[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:33 dist-test-slave-44vj krb5kdc[821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:33 dist-test-slave-44vj krb5kdc[821](info): set up 2 sockets
Jun 19 12:04:33 dist-test-slave-44vj krb5kdc[821](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 12:04:34.617396 814 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:34 dist-test-slave-44vj krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:34.741828 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42975
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:42975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:34.830998 834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:34.831266 834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:34.831300 834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:34.832551 834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:34.832594 834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:34.832612 834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:34.832628 834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:34.832643 834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:34.834225 834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:42975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:34.834580 834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:34.834859 834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:34.840399 839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:34.841198 842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:34.841210 834 server_base.cc:1034] running on GCE node
W20260619 12:04:34.841398 840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:34.841864 834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:34.842271 834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:34.843444 834 hybrid_clock.cc:648] HybridClock initialized: now 1781870674843401 us; error 70 us; skew 500 ppm
Jun 19 12:04:34 dist-test-slave-44vj krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:34.846522 834 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:34.847182 834 webserver.cc:460] Webserver started at http://127.0.203.190:36213/ using document root <none> and password file <none>
I20260619 12:04:34.847491 834 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:34.847548 834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:34.847730 834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:34.848970 834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "4a7ef9be8ade432ba4d399da6f996d4c"
format_stamp: "Formatted at 2026-06-19 12:04:34 on dist-test-slave-44vj"
server_key: "6ea41d318936b82d19c5d81ab553339a"
server_key_iv: "a007c1c999da0be994bfc840bd4f2364"
server_key_version: "encryptionkey@0"
I20260619 12:04:34.849360 834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "4a7ef9be8ade432ba4d399da6f996d4c"
format_stamp: "Formatted at 2026-06-19 12:04:34 on dist-test-slave-44vj"
server_key: "6ea41d318936b82d19c5d81ab553339a"
server_key_iv: "a007c1c999da0be994bfc840bd4f2364"
server_key_version: "encryptionkey@0"
I20260619 12:04:34.851117 834 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 12:04:34.852809 849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:34.853061 834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:34.853169 834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "4a7ef9be8ade432ba4d399da6f996d4c"
format_stamp: "Formatted at 2026-06-19 12:04:34 on dist-test-slave-44vj"
server_key: "6ea41d318936b82d19c5d81ab553339a"
server_key_iv: "a007c1c999da0be994bfc840bd4f2364"
server_key_version: "encryptionkey@0"
I20260619 12:04:34.853251 834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:34.869408 834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:34.872288 834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:34.872539 834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:34.877628 834 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:42975
I20260619 12:04:34.877718 901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:42975 every 8 connection(s)
I20260619 12:04:34.878109 834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:34.879201 902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:34.882321 902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c: Bootstrap starting.
I20260619 12:04:34.883096 902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:34.883443 902 log.cc:824] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:34.884444 902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c: No bootstrap required, opened a new log
I20260619 12:04:34.886168 902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42975 } }
I20260619 12:04:34.886353 902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:34.886389 902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a7ef9be8ade432ba4d399da6f996d4c, State: Initialized, Role: FOLLOWER
I20260619 12:04:34.886547 902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42975 } }
I20260619 12:04:34.886621 902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:34.886656 902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:34.886713 902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:34.887454 902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42975 } }
I20260619 12:04:34.887573 902 leader_election.cc:302] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a7ef9be8ade432ba4d399da6f996d4c; no voters:
I20260619 12:04:34.887776 902 leader_election.cc:288] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:34.887877 905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:34.888123 902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:34.888144 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 834
I20260619 12:04:34.888206 905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [term 1 LEADER]: Becoming Leader. State: Replica: 4a7ef9be8ade432ba4d399da6f996d4c, State: Running, Role: LEADER
I20260619 12:04:34.888240 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:34.888353 905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42975 } }
I20260619 12:04:34.888985 906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42975 } } }
I20260619 12:04:34.889088 906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:34.889122 814 external_mini_cluster.cc:1428] Setting key 448e371ba31c920733eff2309f7919b0
I20260619 12:04:34.889485 907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a7ef9be8ade432ba4d399da6f996d4c. Latest consensus state: current_term: 1 leader_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7ef9be8ade432ba4d399da6f996d4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42975 } } }
I20260619 12:04:34.889559 907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:34.889783 914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:34.890609 914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:34.892413 914 catalog_manager.cc:1347] Generated new cluster ID: b2c47d010032458fac1cfdfc5efb74a0
I20260619 12:04:34.892462 914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:04:34 dist-test-slave-44vj krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:34.906281 914 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:34.907097 914 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:34.910777 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:34.890290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.0.203.190:42975)
0619 12:04:34.890543 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:34.890555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:34.891033 (+ 478us) server_negotiation.cc:407] Connection header received
0619 12:04:34.893784 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:34.893796 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:34.893887 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:34.894046 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:34.896467 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:34.898003 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:34.899742 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:34.900466 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:34.900580 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:34.905629 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:34.905636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:34.905637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:34.905654 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:34.908672 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:34.909204 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:34.909208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:34.909208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:34.909309 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:34.909558 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:34.909559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:34.909559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:34.909837 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:34.909999 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:34.910294 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 12:04:34.910459 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":101,"threads_started":1}
I20260619 12:04:34.912040 914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a7ef9be8ade432ba4d399da6f996d4c: Generated new TSK 0
I20260619 12:04:34.912231 914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:34.941736 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42975
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:35.034528 928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:35.034768 928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:35.034796 928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:35.036224 928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:35.036296 928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:35.036329 928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:35.038095 928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42975
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:35.038398 928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:35.038652 928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:35.044564 934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:35.044564 933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:35.044600 936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:35.044957 928 server_base.cc:1034] running on GCE node
I20260619 12:04:35.045255 928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:35.045625 928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:35.046813 928 hybrid_clock.cc:648] HybridClock initialized: now 1781870675046798 us; error 39 us; skew 500 ppm
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:35.050064 928 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:35.050771 928 webserver.cc:460] Webserver started at http://127.0.203.129:35983/ using document root <none> and password file <none>
I20260619 12:04:35.051036 928 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:35.051100 928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:35.051249 928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:35.053443 928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "a596ce666fb44a1f938ed9077709833b"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "1e1407617e8895d85e119da732c6e8f8"
server_key_iv: "8e3b5aba20396a51e753e8d7e4874fad"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.053874 928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "a596ce666fb44a1f938ed9077709833b"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "1e1407617e8895d85e119da732c6e8f8"
server_key_iv: "8e3b5aba20396a51e753e8d7e4874fad"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.055646 928 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 12:04:35.056751 943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.056917 928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:35.057020 928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "a596ce666fb44a1f938ed9077709833b"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "1e1407617e8895d85e119da732c6e8f8"
server_key_iv: "8e3b5aba20396a51e753e8d7e4874fad"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.057113 928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:35.076171 928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:35.079434 928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:35.079615 928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:35.079916 928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:35.080288 928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:35.080325 928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.080359 928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:35.080384 928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.088791 928 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:45337
I20260619 12:04:35.088963 1056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:45337 every 8 connection(s)
I20260619 12:04:35.089291 928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:35.098613 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 928
I20260619 12:04:35.098731 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:35.098906 814 external_mini_cluster.cc:1428] Setting key 343e2d4b54a2bff2743bb78d18ecc2d2
I20260619 12:04:35.101114 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.090330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:52465 (local address 127.0.203.190:42975)
0619 12:04:35.090468 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:35.090470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:35.090713 (+ 243us) server_negotiation.cc:407] Connection header received
0619 12:04:35.091664 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:35.091665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:35.091678 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:35.091744 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:35.092875 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.093466 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.094639 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.094869 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.094946 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.097585 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:35.097587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:35.097587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:35.097596 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:35.099319 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.099901 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.099902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.099903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.099971 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.100284 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.100285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.100286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.100408 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:35.100464 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:35.100889 (+ 425us) server_negotiation.cc:299] Negotiation successful
0619 12:04:35.100997 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 12:04:35.101709 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.090272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42975 (local address 127.0.203.129:52465)
0619 12:04:35.090476 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:35.090530 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:35.091494 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:35.091834 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:35.091838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:35.092515 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:35.092756 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:35.092763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.093608 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.093609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:35.094521 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:35.094524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.094999 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.095000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:35.095134 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.095797 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:35.095807 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:35.097455 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:35.099465 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:35.099476 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:35.099478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:35.099819 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:35.100049 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:35.100050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:35.100051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:35.100213 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:35.100471 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:35.100476 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:35.100674 (+ 198us) client_negotiation.cc:769] Sending connection context
0619 12:04:35.100805 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 12:04:35.101007 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
I20260619 12:04:35.102358 1057 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42975
I20260619 12:04:35.102532 1057 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:35.102772 1057 heartbeater.cc:511] Master 127.0.203.190:42975 requested a full tablet report, sending...
I20260619 12:04:35.103543 866 ts_manager.cc:194] Registered new tserver with Master: a596ce666fb44a1f938ed9077709833b (127.0.203.129:45337)
I20260619 12:04:35.104240 866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:52465
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:35.126655 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42975
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:35.218415 1062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:35.218685 1062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:35.218713 1062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:35.220064 1062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:35.220120 1062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:35.220151 1062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:35.221752 1062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42975
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:35.222023 1062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:35.222249 1062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:35.227772 1067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:35.228001 1070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:35.228192 1068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:35.228211 1062 server_base.cc:1034] running on GCE node
I20260619 12:04:35.228434 1062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:35.228717 1062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:35.229872 1062 hybrid_clock.cc:648] HybridClock initialized: now 1781870675229855 us; error 37 us; skew 500 ppm
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:35.232734 1062 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:35.233302 1062 webserver.cc:460] Webserver started at http://127.0.203.130:40041/ using document root <none> and password file <none>
I20260619 12:04:35.233522 1062 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:35.233570 1062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:35.233703 1062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:35.234876 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "3d447153f7e8476ea2a1ddf94165c8f1"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "af6e7987bc6bad677649cd196502f963"
server_key_iv: "73cba428004811e9b1c3f5ec1ea17ce2"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.235249 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "3d447153f7e8476ea2a1ddf94165c8f1"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "af6e7987bc6bad677649cd196502f963"
server_key_iv: "73cba428004811e9b1c3f5ec1ea17ce2"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.236804 1062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:35.237658 1077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.237913 1062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:35.238020 1062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "3d447153f7e8476ea2a1ddf94165c8f1"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "af6e7987bc6bad677649cd196502f963"
server_key_iv: "73cba428004811e9b1c3f5ec1ea17ce2"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.238091 1062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:35.247712 1062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:35.250049 1062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:35.250195 1062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:35.250459 1062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:35.250800 1062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:35.250833 1062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.250865 1062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:35.250890 1062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.259865 1062 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:41165
I20260619 12:04:35.260046 1190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:41165 every 8 connection(s)
I20260619 12:04:35.260360 1062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:35.262539 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 1062
I20260619 12:04:35.262653 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:35.262810 814 external_mini_cluster.cc:1428] Setting key 854453ad9641874d5c63e7334f28d349
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
I20260619 12:04:35.272569 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.261381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:53391 (local address 127.0.203.190:42975)
0619 12:04:35.261465 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:35.261468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:35.261684 (+ 216us) server_negotiation.cc:407] Connection header received
0619 12:04:35.262284 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:35.262285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:35.262302 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:35.262418 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:35.263987 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.264686 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.266083 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.266308 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.266379 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.269244 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:35.269247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:35.269247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:35.269256 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:35.270880 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.271400 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.271401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.271402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.271464 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.271724 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.271725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.271725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.271870 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:35.271929 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:35.272357 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 12:04:35.272487 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:04:35.272683 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.261347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42975 (local address 127.0.203.130:53391)
0619 12:04:35.261521 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:35.261576 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:35.262170 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:35.262583 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:35.262588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:35.263536 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:35.263862 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:35.263870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.264861 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.264863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:35.265930 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:35.265935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.266394 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.266395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:35.266579 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.267257 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:35.267267 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:35.269123 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:35.270981 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:35.270989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:35.270993 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:35.271297 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:35.271531 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:35.271532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:35.271533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:35.271669 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:35.271924 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:35.271930 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:35.272093 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 12:04:35.272250 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 12:04:35.272446 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260619 12:04:35.273383 1191 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42975
I20260619 12:04:35.273593 1191 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
I20260619 12:04:35.273821 1191 heartbeater.cc:511] Master 127.0.203.190:42975 requested a full tablet report, sending...
I20260619 12:04:35.274335 866 ts_manager.cc:194] Registered new tserver with Master: 3d447153f7e8476ea2a1ddf94165c8f1 (127.0.203.130:41165)
I20260619 12:04:35.274752 866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:53391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:35.290182 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42975
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:35.382895 1196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:35.383140 1196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:35.383168 1196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:35.384727 1196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:35.384801 1196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:35.384835 1196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:35.386555 1196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42975
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:35.386845 1196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:35.387101 1196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:35.392831 1201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:35.392933 1204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:35.392972 1202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:35.393684 1196 server_base.cc:1034] running on GCE node
I20260619 12:04:35.393946 1196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:35.394238 1196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:35.395463 1196 hybrid_clock.cc:648] HybridClock initialized: now 1781870675395430 us; error 56 us; skew 500 ppm
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:35.398224 1196 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:35.398854 1196 webserver.cc:460] Webserver started at http://127.0.203.131:46673/ using document root <none> and password file <none>
I20260619 12:04:35.399083 1196 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:35.399134 1196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:35.399271 1196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:35.400418 1196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "fc53ad67bbee4ec3accbf2a2a14fde11"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "18fd412bf46839efd1b765152dd26ff0"
server_key_iv: "3a3a8446dfd5a336c4a1773f23cc0849"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.400765 1196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "fc53ad67bbee4ec3accbf2a2a14fde11"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "18fd412bf46839efd1b765152dd26ff0"
server_key_iv: "3a3a8446dfd5a336c4a1773f23cc0849"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.402277 1196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:35.403203 1211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.403492 1196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:35.403620 1196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "fc53ad67bbee4ec3accbf2a2a14fde11"
format_stamp: "Formatted at 2026-06-19 12:04:35 on dist-test-slave-44vj"
server_key: "18fd412bf46839efd1b765152dd26ff0"
server_key_iv: "3a3a8446dfd5a336c4a1773f23cc0849"
server_key_version: "encryptionkey@0"
I20260619 12:04:35.403709 1196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:35.421090 1196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:35.424894 1196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:35.425101 1196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:35.425390 1196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:35.425770 1196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:35.425810 1196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.425836 1196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:35.425858 1196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:35.434902 1196 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:36547
I20260619 12:04:35.434994 1324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:36547 every 8 connection(s)
I20260619 12:04:35.435413 1196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:35.445230 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 1196
I20260619 12:04:35.445322 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:35.445493 814 external_mini_cluster.cc:1428] Setting key 32d76b01de4213c5fb9d4f3f07f845da
I20260619 12:04:35.447211 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.436822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:59689 (local address 127.0.203.190:42975)
0619 12:04:35.436900 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:35.436903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:35.437160 (+ 257us) server_negotiation.cc:407] Connection header received
0619 12:04:35.437796 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:35.437797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:35.437836 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:35.437919 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:35.439073 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.439684 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.440905 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.441120 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.441199 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.443919 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:35.443921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:35.443921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:35.443932 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:35.445318 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.445874 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.445876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.445876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.445984 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.446376 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.446377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.446378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.446558 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:35.446642 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:35.447016 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 12:04:35.447122 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:04:35.447511 1327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.436778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42975 (local address 127.0.203.131:59689)
0619 12:04:35.436985 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:35.437038 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:35.437657 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:35.438000 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:35.438004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:35.438688 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:35.438933 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:35.438939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.439832 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.439832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:35.440795 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:35.440799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.441239 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.441241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:35.441409 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.442060 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:35.442071 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:35.443768 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:35.445424 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:35.445433 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:35.445436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:35.445768 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:35.446082 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:35.446083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:35.446083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:35.446265 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:35.446641 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:35.446648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:35.446841 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 12:04:35.446973 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 12:04:35.447198 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260619 12:04:35.448315 1325 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42975
I20260619 12:04:35.448426 1325 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:35.448624 1325 heartbeater.cc:511] Master 127.0.203.190:42975 requested a full tablet report, sending...
I20260619 12:04:35.449029 866 ts_manager.cc:194] Registered new tserver with Master: fc53ad67bbee4ec3accbf2a2a14fde11 (127.0.203.131:36547)
I20260619 12:04:35.449447 866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:59689
I20260619 12:04:35.458590 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:35.484923 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.475765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.0.203.129:45337)
0619 12:04:35.475934 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:35.475939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:35.475952 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:04:35.476065 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:35.476081 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:35.476126 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:35.476264 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:35.476973 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.477727 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.478623 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.478991 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.479077 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.481779 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:35.481789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:35.481789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:35.481799 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:35.483424 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.483897 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.483904 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.483904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.484005 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.484195 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.484196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.484197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.484457 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:35.484634 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:35.484685 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 12:04:35.484783 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
W20260619 12:04:35.485396 971 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32846
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:35.511796 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:35.503459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.0.203.129:45337)
0619 12:04:35.503576 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:35.503578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:35.503633 (+ 55us) server_negotiation.cc:407] Connection header received
0619 12:04:35.503681 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:35.503682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:35.503699 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:35.503770 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:35.504425 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.505021 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.505882 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:35.506109 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:35.506178 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:35.508905 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:35.508907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:35.508907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:35.508915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:35.510341 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.510934 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.510935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.510936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.511018 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:35.511216 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:35.511217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:35.511217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:35.511324 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:35.511457 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:35.511636 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 12:04:35.511715 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 12:04:35.512513 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 928
I20260619 12:04:35.517654 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 1062
I20260619 12:04:35.522800 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 1196
I20260619 12:04:35.526976 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 834
2026-06-19T12:04:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1908 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[1339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[1339](info): set up 2 sockets
Jun 19 12:04:35 dist-test-slave-44vj krb5kdc[1339](info): commencing operation
krb5kdc: starting...
W20260619 12:04:36.487025 814 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:36 dist-test-slave-44vj krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:36.614233 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:46211
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:46211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:36.702349 1352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:36.702636 1352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:36.702661 1352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:36.704005 1352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:36.704048 1352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:36.704062 1352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:36.704073 1352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:36.704085 1352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:36.705760 1352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:46211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:46211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:36.706012 1352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:36.706216 1352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:36.711834 1357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:36.711930 1352 server_base.cc:1034] running on GCE node
W20260619 12:04:36.713162 1360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:36.713196 1358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:36.713563 1352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:36.713976 1352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:36.715164 1352 hybrid_clock.cc:648] HybridClock initialized: now 1781870676715122 us; error 54 us; skew 500 ppm
Jun 19 12:04:36 dist-test-slave-44vj krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:36.717958 1352 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:36.718521 1352 webserver.cc:460] Webserver started at http://127.0.203.190:38901/ using document root <none> and password file <none>
I20260619 12:04:36.718736 1352 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:36.718783 1352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:36.718900 1352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:36.720105 1352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "665d175f237e466090a1c1dea39c4531"
format_stamp: "Formatted at 2026-06-19 12:04:36 on dist-test-slave-44vj"
server_key: "bc17ffc0201dbb17ad0adcf7fbde14eb"
server_key_iv: "ed23933dc4374373c89954b491560b9d"
server_key_version: "encryptionkey@0"
I20260619 12:04:36.720440 1352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "665d175f237e466090a1c1dea39c4531"
format_stamp: "Formatted at 2026-06-19 12:04:36 on dist-test-slave-44vj"
server_key: "bc17ffc0201dbb17ad0adcf7fbde14eb"
server_key_iv: "ed23933dc4374373c89954b491560b9d"
server_key_version: "encryptionkey@0"
I20260619 12:04:36.721868 1352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:36.722657 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:36.722863 1352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:36.722965 1352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "665d175f237e466090a1c1dea39c4531"
format_stamp: "Formatted at 2026-06-19 12:04:36 on dist-test-slave-44vj"
server_key: "bc17ffc0201dbb17ad0adcf7fbde14eb"
server_key_iv: "ed23933dc4374373c89954b491560b9d"
server_key_version: "encryptionkey@0"
I20260619 12:04:36.723032 1352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:36.733491 1352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:36.735658 1352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:36.735805 1352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:36.740031 1352 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:46211
I20260619 12:04:36.740119 1419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:46211 every 8 connection(s)
I20260619 12:04:36.740449 1352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:36.741463 1420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:36.744445 1420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531: Bootstrap starting.
I20260619 12:04:36.745132 1420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:36.745447 1420 log.cc:824] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:36.746359 1420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531: No bootstrap required, opened a new log
I20260619 12:04:36.748236 1420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665d175f237e466090a1c1dea39c4531" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 46211 } }
I20260619 12:04:36.748401 1420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:36.748423 1420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665d175f237e466090a1c1dea39c4531, State: Initialized, Role: FOLLOWER
I20260619 12:04:36.748575 1420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665d175f237e466090a1c1dea39c4531" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 46211 } }
I20260619 12:04:36.748664 1420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:36.748689 1420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:36.748723 1420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:36.749318 1420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665d175f237e466090a1c1dea39c4531" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 46211 } }
I20260619 12:04:36.749414 1420 leader_election.cc:302] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 665d175f237e466090a1c1dea39c4531; no voters:
I20260619 12:04:36.749606 1420 leader_election.cc:288] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:36.749723 1423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:36.749914 1420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:36.750201 1423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [term 1 LEADER]: Becoming Leader. State: Replica: 665d175f237e466090a1c1dea39c4531, State: Running, Role: LEADER
I20260619 12:04:36.750232 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 1352
I20260619 12:04:36.750308 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:36.750351 1423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665d175f237e466090a1c1dea39c4531" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 46211 } }
I20260619 12:04:36.750496 814 external_mini_cluster.cc:1428] Setting key 963dd5ea0a37913d8720f6ddd1f43ec1
I20260619 12:04:36.751052 1424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "665d175f237e466090a1c1dea39c4531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665d175f237e466090a1c1dea39c4531" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 46211 } } }
I20260619 12:04:36.751165 1424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:36.751664 1425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 665d175f237e466090a1c1dea39c4531. Latest consensus state: current_term: 1 leader_uuid: "665d175f237e466090a1c1dea39c4531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665d175f237e466090a1c1dea39c4531" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 46211 } } }
I20260619 12:04:36.751922 1425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531 [sys.catalog]: This master's current role is: LEADER
W20260619 12:04:36.752188 1439 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:04:36.752238 1439 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:04:36.752298 1438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:36.752545 1438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:36.754245 1438 catalog_manager.cc:1347] Generated new cluster ID: 65d06540aa954607ac8763201c8f24de
I20260619 12:04:36.754310 1438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:04:36 dist-test-slave-44vj krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:36.761792 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:36.751338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.0.203.190:46211)
0619 12:04:36.751808 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:36.751820 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:36.751864 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:04:36.752135 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:36.752145 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:36.752213 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:36.752415 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:36.753066 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:36.754689 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:36.755560 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:36.756033 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:36.756148 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:36.758533 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:36.758538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:36.758538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:36.758553 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:36.760261 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:36.760591 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:36.760594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:36.760595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:36.760702 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:36.760873 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:36.760874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:36.760875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:36.761162 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:36.761325 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:36.761407 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 12:04:36.761569 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":68,"threads_started":1}
I20260619 12:04:36.776957 1438 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:36.777762 1438 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:36.781042 1438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 665d175f237e466090a1c1dea39c4531: Generated new TSK 0
I20260619 12:04:36.781266 1438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:36.814186 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:46211
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:36.903788 1446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:36.904052 1446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:36.904086 1446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:36.905364 1446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:36.905405 1446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:36.905442 1446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:36.907020 1446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:46211
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:36.907292 1446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:36.907573 1446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:36.913311 1451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:36.913478 1446 server_base.cc:1034] running on GCE node
W20260619 12:04:36.913338 1454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:36.913311 1452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:36.913846 1446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:36.914153 1446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:36.915324 1446 hybrid_clock.cc:648] HybridClock initialized: now 1781870676915299 us; error 48 us; skew 500 ppm
Jun 19 12:04:36 dist-test-slave-44vj krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:36.918193 1446 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:36.918787 1446 webserver.cc:460] Webserver started at http://127.0.203.129:43061/ using document root <none> and password file <none>
I20260619 12:04:36.919059 1446 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:36.919113 1446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:36.919250 1446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:36.920482 1446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "6f4beec9a66846bea71eb2da808cd26b"
format_stamp: "Formatted at 2026-06-19 12:04:36 on dist-test-slave-44vj"
server_key: "b883ad42438b1da22e59a53aa2be67ae"
server_key_iv: "ee60059e37cf77474eb7f202f69c73d7"
server_key_version: "encryptionkey@0"
I20260619 12:04:36.920872 1446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "6f4beec9a66846bea71eb2da808cd26b"
format_stamp: "Formatted at 2026-06-19 12:04:36 on dist-test-slave-44vj"
server_key: "b883ad42438b1da22e59a53aa2be67ae"
server_key_iv: "ee60059e37cf77474eb7f202f69c73d7"
server_key_version: "encryptionkey@0"
I20260619 12:04:36.922355 1446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:36.923264 1461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:36.923513 1446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:36.923632 1446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "6f4beec9a66846bea71eb2da808cd26b"
format_stamp: "Formatted at 2026-06-19 12:04:36 on dist-test-slave-44vj"
server_key: "b883ad42438b1da22e59a53aa2be67ae"
server_key_iv: "ee60059e37cf77474eb7f202f69c73d7"
server_key_version: "encryptionkey@0"
I20260619 12:04:36.923708 1446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:36.940014 1446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:36.942116 1446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:36.942255 1446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:36.942480 1446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:36.942804 1446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:36.942842 1446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:36.942871 1446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:36.942888 1446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:36.949224 1446 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:39201
I20260619 12:04:36.949321 1574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:39201 every 8 connection(s)
I20260619 12:04:36.949635 1446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:04:36.950224 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 1446
I20260619 12:04:36.950327 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:36.950538 814 external_mini_cluster.cc:1428] Setting key 92a9876869a1378804738f1088944d84
Jun 19 12:04:36 dist-test-slave-44vj krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
I20260619 12:04:36.961441 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:36.950886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:57129 (local address 127.0.203.190:46211)
0619 12:04:36.950948 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:36.950950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:36.951332 (+ 382us) server_negotiation.cc:407] Connection header received
0619 12:04:36.952496 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:36.952497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:36.952517 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:36.952616 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:36.953594 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:36.954162 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:36.955271 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:36.955574 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:36.955638 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:36.958049 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:36.958051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:36.958052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:36.958061 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:36.959658 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:36.960289 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:36.960291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:36.960291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:36.960360 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:36.960674 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:36.960675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:36.960675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:36.960815 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:36.960881 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:36.961225 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 12:04:36.961330 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:04:36.961743 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:36.950890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:46211 (local address 127.0.203.129:57129)
0619 12:04:36.951123 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:36.951202 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:36.952367 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:36.952839 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:36.952845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:36.953206 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:36.953469 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:36.953475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:36.954253 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:36.954254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:36.955186 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:36.955189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:36.955657 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:36.955658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:36.955797 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:36.956416 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:36.956426 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:36.957946 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:36.959772 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:36.959786 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:36.959789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:36.960162 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:36.960440 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:36.960441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:36.960441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:36.960592 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:36.960894 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:36.960904 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:36.961084 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 12:04:36.961195 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 12:04:36.961413 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260619 12:04:36.962425 1575 heartbeater.cc:347] Connected to a master server at 127.0.203.190:46211
I20260619 12:04:36.962555 1575 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:36.962788 1575 heartbeater.cc:511] Master 127.0.203.190:46211 requested a full tablet report, sending...
I20260619 12:04:36.963474 1384 ts_manager.cc:194] Registered new tserver with Master: 6f4beec9a66846bea71eb2da808cd26b (127.0.203.129:39201)
I20260619 12:04:36.964148 1384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:57129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:36.978616 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:46211
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:37.068769 1580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:37.069015 1580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:37.069042 1580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:37.070350 1580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:37.070413 1580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:37.070447 1580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:37.072142 1580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:46211
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:37.072427 1580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:37.072665 1580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:37.078346 1586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:37.078392 1588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:37.078346 1585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:37.078640 1580 server_base.cc:1034] running on GCE node
I20260619 12:04:37.078912 1580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:37.079193 1580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:37.080369 1580 hybrid_clock.cc:648] HybridClock initialized: now 1781870677080350 us; error 41 us; skew 500 ppm
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:37.083096 1580 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:37.083701 1580 webserver.cc:460] Webserver started at http://127.0.203.130:40163/ using document root <none> and password file <none>
I20260619 12:04:37.083950 1580 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:37.084002 1580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:37.084141 1580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:37.085223 1580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "107f1562abf8467ca5c5ece17e73276f"
format_stamp: "Formatted at 2026-06-19 12:04:37 on dist-test-slave-44vj"
server_key: "2a21e1742a9f0fb5d938419aa8c74b29"
server_key_iv: "fdfa92591433b671d655033c94cd4bcf"
server_key_version: "encryptionkey@0"
I20260619 12:04:37.085526 1580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "107f1562abf8467ca5c5ece17e73276f"
format_stamp: "Formatted at 2026-06-19 12:04:37 on dist-test-slave-44vj"
server_key: "2a21e1742a9f0fb5d938419aa8c74b29"
server_key_iv: "fdfa92591433b671d655033c94cd4bcf"
server_key_version: "encryptionkey@0"
I20260619 12:04:37.086810 1580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:37.087635 1595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:37.088039 1580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:37.088140 1580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "107f1562abf8467ca5c5ece17e73276f"
format_stamp: "Formatted at 2026-06-19 12:04:37 on dist-test-slave-44vj"
server_key: "2a21e1742a9f0fb5d938419aa8c74b29"
server_key_iv: "fdfa92591433b671d655033c94cd4bcf"
server_key_version: "encryptionkey@0"
I20260619 12:04:37.088222 1580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:37.109903 1580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:37.112156 1580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:37.112308 1580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:37.112542 1580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:37.112860 1580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:37.112897 1580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:37.112928 1580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:37.112944 1580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:37.119529 1580 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:35943
I20260619 12:04:37.119625 1708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:35943 every 8 connection(s)
I20260619 12:04:37.119944 1580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:37.125057 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 1580
I20260619 12:04:37.125159 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:37.125327 814 external_mini_cluster.cc:1428] Setting key 000bcb5e00b5259ff3126bb082ed6103
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:37.131039 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.120851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:44367 (local address 127.0.203.190:46211)
0619 12:04:37.120938 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.120947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.121152 (+ 205us) server_negotiation.cc:407] Connection header received
0619 12:04:37.121781 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.121783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.121798 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.121854 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:37.122946 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.123508 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.124582 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.124779 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.124839 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.127690 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:37.127692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:37.127693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:37.127703 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:37.129211 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.129722 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.129723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.129723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.129801 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.130166 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.130168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.130168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.130333 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:37.130427 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.130840 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.130958 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:04:37.131278 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.120750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:46211 (local address 127.0.203.130:44367)
0619 12:04:37.120970 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:37.121049 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:37.121673 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:37.121935 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:37.121940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:37.122613 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:37.122850 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.122855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.123597 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.123598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.124498 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.124500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.124852 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.124853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.124992 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.125795 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:37.125807 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:37.127530 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:37.129311 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:37.129318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:37.129321 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:37.129640 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:37.129895 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:37.129896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:37.129896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:37.130060 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:37.130425 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:37.130431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:37.130637 (+ 206us) client_negotiation.cc:769] Sending connection context
0619 12:04:37.130778 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 12:04:37.130981 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260619 12:04:37.131907 1709 heartbeater.cc:347] Connected to a master server at 127.0.203.190:46211
I20260619 12:04:37.132048 1709 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:37.132269 1709 heartbeater.cc:511] Master 127.0.203.190:46211 requested a full tablet report, sending...
I20260619 12:04:37.132730 1384 ts_manager.cc:194] Registered new tserver with Master: 107f1562abf8467ca5c5ece17e73276f (127.0.203.130:35943)
I20260619 12:04:37.133131 1384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:44367
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:37.152089 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:46211
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:37.241571 1714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:37.241855 1714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:37.241887 1714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:37.243250 1714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:37.243300 1714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:37.243331 1714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:37.245249 1714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:46211
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:37.245532 1714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:37.245783 1714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:37.251475 1722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:37.251489 1720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:37.251824 1714 server_base.cc:1034] running on GCE node
W20260619 12:04:37.251475 1719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:37.252128 1714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:37.252441 1714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:37.253619 1714 hybrid_clock.cc:648] HybridClock initialized: now 1781870677253594 us; error 54 us; skew 500 ppm
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:37.259171 1714 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:37.259739 1714 webserver.cc:460] Webserver started at http://127.0.203.131:40151/ using document root <none> and password file <none>
I20260619 12:04:37.259984 1714 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:37.260036 1714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:37.260174 1714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:37.261567 1714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "6edbf3c447554fbdbb40a829369d5c3e"
format_stamp: "Formatted at 2026-06-19 12:04:37 on dist-test-slave-44vj"
server_key: "a896739be637a848d66d22ffbc280f56"
server_key_iv: "8b462c53c52888846792af74985355cf"
server_key_version: "encryptionkey@0"
I20260619 12:04:37.262006 1714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "6edbf3c447554fbdbb40a829369d5c3e"
format_stamp: "Formatted at 2026-06-19 12:04:37 on dist-test-slave-44vj"
server_key: "a896739be637a848d66d22ffbc280f56"
server_key_iv: "8b462c53c52888846792af74985355cf"
server_key_version: "encryptionkey@0"
I20260619 12:04:37.263550 1714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:37.264570 1729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:37.264810 1714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:37.264919 1714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "6edbf3c447554fbdbb40a829369d5c3e"
format_stamp: "Formatted at 2026-06-19 12:04:37 on dist-test-slave-44vj"
server_key: "a896739be637a848d66d22ffbc280f56"
server_key_iv: "8b462c53c52888846792af74985355cf"
server_key_version: "encryptionkey@0"
I20260619 12:04:37.265015 1714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:37.273420 1714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:37.276163 1714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:37.276301 1714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:37.276543 1714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:37.276889 1714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:37.276925 1714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:37.276948 1714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:37.276959 1714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:37.284288 1714 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:34827
I20260619 12:04:37.284395 1842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:34827 every 8 connection(s)
I20260619 12:04:37.284725 1714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:04:37.287724 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 1714
I20260619 12:04:37.287839 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:37.287999 814 external_mini_cluster.cc:1428] Setting key 82bc59b1cc1d8262fc4708d59602257c
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:37.295904 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.285669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:42001 (local address 127.0.203.190:46211)
0619 12:04:37.285756 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.285758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.285994 (+ 236us) server_negotiation.cc:407] Connection header received
0619 12:04:37.286571 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.286573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.286585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.286645 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:37.287936 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.288760 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.289887 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.290092 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.290153 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.292703 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:37.292704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:37.292705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:37.292713 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:37.294159 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.294661 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.294661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.294662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.294714 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.295005 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.295007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.295007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.295160 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:37.295239 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.295725 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.295818 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:04:37.296039 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.285608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:46211 (local address 127.0.203.131:42001)
0619 12:04:37.285824 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:37.285872 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:37.286449 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:37.286725 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:37.286730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:37.287449 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:37.287757 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.287765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.288846 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.288847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.289794 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.289797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.290167 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.290168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.290272 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.290850 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:37.290860 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:37.292592 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:37.294260 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:37.294268 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:37.294270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:37.294590 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:37.294779 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:37.294779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:37.294779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:37.294895 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:37.295247 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:37.295253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:37.295541 (+ 288us) client_negotiation.cc:769] Sending connection context
0619 12:04:37.295649 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 12:04:37.295813 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
I20260619 12:04:37.296651 1843 heartbeater.cc:347] Connected to a master server at 127.0.203.190:46211
I20260619 12:04:37.296802 1843 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:37.297075 1843 heartbeater.cc:511] Master 127.0.203.190:46211 requested a full tablet report, sending...
I20260619 12:04:37.297562 1384 ts_manager.cc:194] Registered new tserver with Master: 6edbf3c447554fbdbb40a829369d5c3e (127.0.203.131:34827)
I20260619 12:04:37.298074 1384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:42001
I20260619 12:04:37.300194 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:04:37.308810 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.301906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.0.203.190:46211)
0619 12:04:37.301975 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.301977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.302201 (+ 224us) server_negotiation.cc:407] Connection header received
0619 12:04:37.302274 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.302274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.302285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.302330 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:37.302897 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.303510 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.304337 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.304533 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.304602 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.305939 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:37.305941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:37.305941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:37.305947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:37.307301 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.307776 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.307777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.307778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.307843 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.308082 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.308083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.308083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.308274 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:37.308354 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.308649 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.308726 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:04:37.311697 1384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:37.313361 1384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:37.321319 1855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.316581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39201 (local address 127.0.0.1:51558)
0619 12:04:37.316795 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:37.316840 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:37.316922 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:37.317778 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:37.317778 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:37.317780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:37.317827 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.317829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.319017 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.319018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.320082 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.320084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.320935 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.320936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.321039 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.321058 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:04:37.321153 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 12:04:37.321226 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":69,"threads_started":1}
I20260619 12:04:37.321329 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.316922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51558 (local address 127.0.203.129:39201)
0619 12:04:37.317363 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.317368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.317378 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:04:37.317428 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.317430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.317447 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.317524 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:37.317907 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.318913 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.320158 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.320831 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.320911 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.320977 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.321179 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.321228 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":66,"threads_started":1}
I20260619 12:04:37.321588 1854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.316597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35943 (local address 127.0.0.1:40828)
0619 12:04:37.317629 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:37.317638 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:37.317701 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:37.317887 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:37.317889 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:37.317892 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:37.317974 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.317976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.318890 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.318891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.320326 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.320330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.321314 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.321315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.321381 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.321386 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:04:37.321435 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:04:37.321506 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":31,"thread_start_us":124,"threads_started":1}
I20260619 12:04:37.321839 1859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.317099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.0.203.130:35943)
0619 12:04:37.317485 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.317491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.317693 (+ 202us) server_negotiation.cc:407] Connection header received
0619 12:04:37.317777 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.317779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.317799 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.317870 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:37.318039 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.318766 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.320434 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.321152 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.321222 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.321275 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.321705 (+ 430us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.321761 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":116,"threads_started":1}
I20260619 12:04:37.322269 1856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.316587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:34827 (local address 127.0.0.1:54200)
0619 12:04:37.316795 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:37.316840 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:37.316922 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:37.317272 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:37.317276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:37.317279 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:37.317379 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.317381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.318812 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.318814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.320342 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.320344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.322013 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.322014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.322092 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.322097 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:04:37.322150 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:04:37.322204 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":57,"thread_start_us":133,"threads_started":1}
I20260619 12:04:37.322553 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.316876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.0.203.131:34827)
0619 12:04:37.317066 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.317071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.317085 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:04:37.317129 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.317131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.317145 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.317246 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:37.317501 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.318689 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.320455 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.321089 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.321162 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.321225 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.322382 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.322457 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":79,"threads_started":1}
I20260619 12:04:37.322433 1509 tablet_service.cc:1467] Processing CreateTablet for tablet 5c38d897fed24d4f924591ad6ff386e4 (DEFAULT_TABLE table=test-table [id=1707a033330c41b8b576713362d62158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:37.322569 1643 tablet_service.cc:1467] Processing CreateTablet for tablet 5c38d897fed24d4f924591ad6ff386e4 (DEFAULT_TABLE table=test-table [id=1707a033330c41b8b576713362d62158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:37.322994 1509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c38d897fed24d4f924591ad6ff386e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:37.323005 1643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c38d897fed24d4f924591ad6ff386e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:37.323179 1777 tablet_service.cc:1467] Processing CreateTablet for tablet 5c38d897fed24d4f924591ad6ff386e4 (DEFAULT_TABLE table=test-table [id=1707a033330c41b8b576713362d62158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:37.323552 1777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c38d897fed24d4f924591ad6ff386e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:37.326167 1860 tablet_bootstrap.cc:492] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b: Bootstrap starting.
I20260619 12:04:37.327042 1860 tablet_bootstrap.cc:654] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:37.327672 1860 log.cc:824] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:37.327924 1862 tablet_bootstrap.cc:492] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e: Bootstrap starting.
I20260619 12:04:37.328603 1860 tablet_bootstrap.cc:492] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b: No bootstrap required, opened a new log
I20260619 12:04:37.328603 1862 tablet_bootstrap.cc:654] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:37.328675 1860 ts_tablet_manager.cc:1397] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:04:37.329125 1862 log.cc:824] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:37.329456 1861 tablet_bootstrap.cc:492] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f: Bootstrap starting.
I20260619 12:04:37.330149 1861 tablet_bootstrap.cc:654] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:37.330715 1861 log.cc:824] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:37.330729 1860 raft_consensus.cc:348] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.330922 1860 raft_consensus.cc:374] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:37.330977 1860 raft_consensus.cc:729] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4beec9a66846bea71eb2da808cd26b, State: Initialized, Role: FOLLOWER
I20260619 12:04:37.331115 1860 consensus_queue.cc:260] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.331334 1862 tablet_bootstrap.cc:492] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e: No bootstrap required, opened a new log
I20260619 12:04:37.331441 1862 ts_tablet_manager.cc:1397] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:04:37.331462 1860 ts_tablet_manager.cc:1428] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:37.331583 1575 heartbeater.cc:503] Master 127.0.203.190:46211 was elected leader, sending a full tablet report...
I20260619 12:04:37.332033 1861 tablet_bootstrap.cc:492] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f: No bootstrap required, opened a new log
I20260619 12:04:37.332114 1861 ts_tablet_manager.cc:1397] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:04:37.333949 1862 raft_consensus.cc:348] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.334162 1862 raft_consensus.cc:374] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:37.334196 1862 raft_consensus.cc:729] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6edbf3c447554fbdbb40a829369d5c3e, State: Initialized, Role: FOLLOWER
I20260619 12:04:37.334151 1861 raft_consensus.cc:348] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.334317 1861 raft_consensus.cc:374] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:37.334359 1861 raft_consensus.cc:729] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 107f1562abf8467ca5c5ece17e73276f, State: Initialized, Role: FOLLOWER
I20260619 12:04:37.334326 1862 consensus_queue.cc:260] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.334470 1861 consensus_queue.cc:260] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.334630 1862 ts_tablet_manager.cc:1428] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:37.334717 1861 ts_tablet_manager.cc:1428] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:37.334723 1843 heartbeater.cc:503] Master 127.0.203.190:46211 was elected leader, sending a full tablet report...
I20260619 12:04:37.334796 1709 heartbeater.cc:503] Master 127.0.203.190:46211 was elected leader, sending a full tablet report...
W20260619 12:04:37.370605 1710 tablet.cc:2369] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:37.447248 1868 raft_consensus.cc:482] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:37.447463 1868 raft_consensus.cc:504] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.447897 1868 leader_election.cc:288] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6edbf3c447554fbdbb40a829369d5c3e (127.0.203.131:34827), 6f4beec9a66846bea71eb2da808cd26b (127.0.203.129:39201)
W20260619 12:04:37.450697 1576 tablet.cc:2369] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:37.451663 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.448353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:55347 (local address 127.0.203.131:34827)
0619 12:04:37.448402 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.448405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.448419 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:04:37.448447 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.448448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.448456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.448538 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:37.448748 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.449325 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.450585 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.451272 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.451335 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.451344 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.451563 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.451598 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:04:37.451695 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.448230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:34827 (local address 127.0.203.130:55347)
0619 12:04:37.448275 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:37.448289 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:37.448341 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:37.448563 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:37.448564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:37.448566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:37.448656 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.448658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.449430 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.449431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.450485 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.450488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.451351 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.451352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.451476 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.451482 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:04:37.451538 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:04:37.451591 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":1}
I20260619 12:04:37.452167 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.448310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:60809 (local address 127.0.203.129:39201)
0619 12:04:37.448391 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.448393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.448730 (+ 337us) server_negotiation.cc:407] Connection header received
0619 12:04:37.448773 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.448774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.448786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.448827 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:37.449034 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.449714 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.451222 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.451790 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.451861 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.451871 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.452070 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.452112 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:04:37.452085 1797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c38d897fed24d4f924591ad6ff386e4" candidate_uuid: "107f1562abf8467ca5c5ece17e73276f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6edbf3c447554fbdbb40a829369d5c3e" is_pre_election: true
I20260619 12:04:37.452168 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.448230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39201 (local address 127.0.203.130:60809)
0619 12:04:37.448647 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:37.448660 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:37.448721 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:37.448849 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:37.448850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:37.448853 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:37.448935 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.448937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.449838 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.449839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.451105 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:37.451110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.451905 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.451906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:37.451988 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.451993 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:04:37.452052 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 12:04:37.452100 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":163,"thread_start_us":129,"threads_started":1}
I20260619 12:04:37.452298 1797 raft_consensus.cc:2454] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 107f1562abf8467ca5c5ece17e73276f in term 0.
I20260619 12:04:37.452666 1599 leader_election.cc:302] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 107f1562abf8467ca5c5ece17e73276f, 6edbf3c447554fbdbb40a829369d5c3e; no voters:
I20260619 12:04:37.452766 1529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c38d897fed24d4f924591ad6ff386e4" candidate_uuid: "107f1562abf8467ca5c5ece17e73276f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4beec9a66846bea71eb2da808cd26b" is_pre_election: true
I20260619 12:04:37.452934 1529 raft_consensus.cc:2454] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 107f1562abf8467ca5c5ece17e73276f in term 0.
I20260619 12:04:37.452934 1868 raft_consensus.cc:2781] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:37.453022 1868 raft_consensus.cc:482] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:37.453060 1868 raft_consensus.cc:3037] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:37.453936 1868 raft_consensus.cc:504] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.454102 1868 leader_election.cc:288] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [CANDIDATE]: Term 1 election: Requested vote from peers 6edbf3c447554fbdbb40a829369d5c3e (127.0.203.131:34827), 6f4beec9a66846bea71eb2da808cd26b (127.0.203.129:39201)
I20260619 12:04:37.454246 1529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c38d897fed24d4f924591ad6ff386e4" candidate_uuid: "107f1562abf8467ca5c5ece17e73276f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4beec9a66846bea71eb2da808cd26b"
I20260619 12:04:37.454309 1529 raft_consensus.cc:3037] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:37.454382 1797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c38d897fed24d4f924591ad6ff386e4" candidate_uuid: "107f1562abf8467ca5c5ece17e73276f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6edbf3c447554fbdbb40a829369d5c3e"
I20260619 12:04:37.454514 1797 raft_consensus.cc:3037] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:37.455113 1529 raft_consensus.cc:2454] T 5c38d897fed24d4f924591ad6ff386e4 P 6f4beec9a66846bea71eb2da808cd26b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 107f1562abf8467ca5c5ece17e73276f in term 1.
I20260619 12:04:37.455322 1596 leader_election.cc:302] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 107f1562abf8467ca5c5ece17e73276f, 6f4beec9a66846bea71eb2da808cd26b; no voters:
I20260619 12:04:37.455530 1797 raft_consensus.cc:2454] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 107f1562abf8467ca5c5ece17e73276f in term 1.
I20260619 12:04:37.455515 1868 raft_consensus.cc:2781] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:37.455813 1868 raft_consensus.cc:686] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [term 1 LEADER]: Becoming Leader. State: Replica: 107f1562abf8467ca5c5ece17e73276f, State: Running, Role: LEADER
I20260619 12:04:37.455935 1868 consensus_queue.cc:237] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } }
I20260619 12:04:37.457186 1384 catalog_manager.cc:5495] T 5c38d897fed24d4f924591ad6ff386e4 P 107f1562abf8467ca5c5ece17e73276f reported cstate change: term changed from 0 to 1, leader changed from <none> to 107f1562abf8467ca5c5ece17e73276f (127.0.203.130). New cstate: current_term: 1 leader_uuid: "107f1562abf8467ca5c5ece17e73276f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "107f1562abf8467ca5c5ece17e73276f" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6edbf3c447554fbdbb40a829369d5c3e" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4beec9a66846bea71eb2da808cd26b" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39201 } health_report { overall_health: UNKNOWN } } }
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:37.473860 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.464051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.0.203.129:39201)
0619 12:04:37.464105 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.464107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.464121 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:04:37.464153 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.464154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.464163 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.464224 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:37.464995 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.465872 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.466849 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.467121 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.467229 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.470133 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:37.470160 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:37.470160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:37.470170 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:37.472253 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.472777 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.472781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.472781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.472856 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.473099 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.473101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.473101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.473317 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:37.473476 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.473670 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.473764 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:37.501034 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.492969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.0.203.129:39201)
0619 12:04:37.493078 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.493080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.493180 (+ 100us) server_negotiation.cc:407] Connection header received
0619 12:04:37.493220 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.493222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.493235 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.493310 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:37.494001 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.494503 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.495311 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.495578 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.495658 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.497998 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:37.498000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:37.498000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:37.498010 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:37.499541 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.499934 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.499935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.499936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.500004 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.500203 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.500204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.500205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.500323 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:37.500416 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.500778 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.500886 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:37.528695 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:37.519423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51576 (local address 127.0.203.129:39201)
0619 12:04:37.519499 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:37.519501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:37.519603 (+ 102us) server_negotiation.cc:407] Connection header received
0619 12:04:37.519686 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:37.519687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:37.519697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:37.519741 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:37.520427 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.520988 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.521900 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:37.522138 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:37.522250 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:37.524989 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:37.524991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:37.524991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:37.525002 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:37.526612 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.527329 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.527330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.527330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.527448 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:37.527857 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:37.527859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:37.527859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:37.528083 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:37.528207 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:37.528432 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 12:04:37.528519 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 12:04:37.529332 1489 tablet_service.cc:2908] Rejecting scan request for tablet 5c38d897fed24d4f924591ad6ff386e4: Uninitialized: clean time has not yet been initialized
I20260619 12:04:37.530493 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 1446
W20260619 12:04:37.535422 1844 tablet.cc:2369] T 5c38d897fed24d4f924591ad6ff386e4 P 6edbf3c447554fbdbb40a829369d5c3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:37.536041 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 1580
I20260619 12:04:37.540923 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 1714
I20260619 12:04:37.546299 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 1352
2026-06-19T12:04:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2020 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1884](info): set up 2 sockets
Jun 19 12:04:37 dist-test-slave-44vj krb5kdc[1884](info): commencing operation
krb5kdc: starting...
W20260619 12:04:38.515152 814 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:38 dist-test-slave-44vj krb5kdc[1884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:38.638640 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:40467
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:40467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:38.731113 1897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:38.731492 1897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:38.731536 1897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:38.733237 1897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:38.733438 1897 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:04:38.733480 1897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:38.733518 1897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:38.733559 1897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:38.733579 1897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:38.736470 1897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:40467
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:40467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:38.736882 1897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:38.737164 1897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:38.743592 1902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:38.743727 1903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:38.743700 1897 server_base.cc:1034] running on GCE node
W20260619 12:04:38.743857 1905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:38.744239 1897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:38.744637 1897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:38.745848 1897 hybrid_clock.cc:648] HybridClock initialized: now 1781870678745774 us; error 104 us; skew 500 ppm
Jun 19 12:04:38 dist-test-slave-44vj krb5kdc[1884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:38.753893 1897 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:38.754591 1897 webserver.cc:460] Webserver started at http://127.0.203.190:36979/ using document root <none> and password file <none>
I20260619 12:04:38.754832 1897 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:38.754886 1897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:38.755023 1897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:38.756242 1897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "2e5949bd0b794cdeb3790f048e5d6223"
format_stamp: "Formatted at 2026-06-19 12:04:38 on dist-test-slave-44vj"
server_key: "7b841ec8fe4393ddc576647c056d6dbe"
server_key_iv: "396781fe6af64497ba7848be8cf73f12"
server_key_version: "encryptionkey@0"
I20260619 12:04:38.756633 1897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "2e5949bd0b794cdeb3790f048e5d6223"
format_stamp: "Formatted at 2026-06-19 12:04:38 on dist-test-slave-44vj"
server_key: "7b841ec8fe4393ddc576647c056d6dbe"
server_key_iv: "396781fe6af64497ba7848be8cf73f12"
server_key_version: "encryptionkey@0"
I20260619 12:04:38.758205 1897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:38.759183 1912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:38.759539 1897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:38.759773 1897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "2e5949bd0b794cdeb3790f048e5d6223"
format_stamp: "Formatted at 2026-06-19 12:04:38 on dist-test-slave-44vj"
server_key: "7b841ec8fe4393ddc576647c056d6dbe"
server_key_iv: "396781fe6af64497ba7848be8cf73f12"
server_key_version: "encryptionkey@0"
I20260619 12:04:38.759895 1897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:38.775578 1897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:38.778163 1897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:38.778337 1897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:38.784691 1897 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:40467
I20260619 12:04:38.784839 1974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:40467 every 8 connection(s)
I20260619 12:04:38.785211 1897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:38.786182 1975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:38.789647 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223: Bootstrap starting.
I20260619 12:04:38.790527 1975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:38.790900 1975 log.cc:824] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:38.792112 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223: No bootstrap required, opened a new log
I20260619 12:04:38.794963 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 1897
I20260619 12:04:38.795065 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:38.794895 1975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5949bd0b794cdeb3790f048e5d6223" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40467 } }
I20260619 12:04:38.795164 1975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:38.795215 1975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5949bd0b794cdeb3790f048e5d6223, State: Initialized, Role: FOLLOWER
I20260619 12:04:38.795257 814 external_mini_cluster.cc:1428] Setting key 51ae34e2d469b9f7ef5c4e562f474794
I20260619 12:04:38.795464 1975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5949bd0b794cdeb3790f048e5d6223" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40467 } }
I20260619 12:04:38.795605 1975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:38.795672 1975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:38.795737 1975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:38.796728 1975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5949bd0b794cdeb3790f048e5d6223" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40467 } }
I20260619 12:04:38.796872 1975 leader_election.cc:302] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e5949bd0b794cdeb3790f048e5d6223; no voters:
I20260619 12:04:38.797073 1975 leader_election.cc:288] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:38.797159 1980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:38.797430 1975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:38.797487 1980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [term 1 LEADER]: Becoming Leader. State: Replica: 2e5949bd0b794cdeb3790f048e5d6223, State: Running, Role: LEADER
I20260619 12:04:38.797624 1980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5949bd0b794cdeb3790f048e5d6223" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40467 } }
I20260619 12:04:38.798177 1982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e5949bd0b794cdeb3790f048e5d6223. Latest consensus state: current_term: 1 leader_uuid: "2e5949bd0b794cdeb3790f048e5d6223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5949bd0b794cdeb3790f048e5d6223" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40467 } } }
I20260619 12:04:38.798296 1982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:38.798727 1981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e5949bd0b794cdeb3790f048e5d6223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5949bd0b794cdeb3790f048e5d6223" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40467 } } }
I20260619 12:04:38.798815 1981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:38.799152 1989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:38.799871 1989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:38.801744 1989 catalog_manager.cc:1347] Generated new cluster ID: f1bdda27cc644a7f80df4b24eb05e25f
I20260619 12:04:38.803112 1989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:04:38 dist-test-slave-44vj krb5kdc[1884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:38.811085 1989 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:38.811946 1989 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:38.812160 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:38.796035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.0.203.190:40467)
0619 12:04:38.796325 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:38.796338 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:38.796379 (+ 41us) server_negotiation.cc:407] Connection header received
0619 12:04:38.796519 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:38.796527 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:38.796581 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:38.796720 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:38.797542 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:38.798991 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:38.800103 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:38.801061 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:38.801264 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:38.804989 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:38.804999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:38.804999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:38.805021 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:38.809400 (+ 4379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:38.810168 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:38.810174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:38.810174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:38.810374 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:38.810695 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:38.810696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:38.810696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:38.811093 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:38.811301 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:38.811527 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 12:04:38.811863 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":129,"threads_started":1}
I20260619 12:04:38.821632 1989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223: Generated new TSK 0
I20260619 12:04:38.821874 1989 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:38.855059 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:40467
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:38.947216 2001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:38.947513 2001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:38.947546 2001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:38.948885 2001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:38.948948 2001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:38.948972 2001 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:04:38.948990 2001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:38.950749 2001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:40467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:38.951017 2001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:38.951256 2001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:38.956586 2006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:38.956568 2007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:38.956768 2001 server_base.cc:1034] running on GCE node
W20260619 12:04:38.956573 2009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:38.957041 2001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:38.957294 2001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:38.958446 2001 hybrid_clock.cc:648] HybridClock initialized: now 1781870678958418 us; error 46 us; skew 500 ppm
Jun 19 12:04:38 dist-test-slave-44vj krb5kdc[1884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:38.961232 2001 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:38.961817 2001 webserver.cc:460] Webserver started at http://127.0.203.129:35197/ using document root <none> and password file <none>
I20260619 12:04:38.962100 2001 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:38.962153 2001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:38.962329 2001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:38.963667 2001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "c6d719b125444d189d4880f4119d295d"
format_stamp: "Formatted at 2026-06-19 12:04:38 on dist-test-slave-44vj"
server_key: "b83d590c19b28bab5cfeeca7e7101b5e"
server_key_iv: "0b75a766065df2080786f24fcc3a4ee5"
server_key_version: "encryptionkey@0"
I20260619 12:04:38.964037 2001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "c6d719b125444d189d4880f4119d295d"
format_stamp: "Formatted at 2026-06-19 12:04:38 on dist-test-slave-44vj"
server_key: "b83d590c19b28bab5cfeeca7e7101b5e"
server_key_iv: "0b75a766065df2080786f24fcc3a4ee5"
server_key_version: "encryptionkey@0"
I20260619 12:04:38.965550 2001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:38.966531 2016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:38.966923 2001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:38.967140 2001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "c6d719b125444d189d4880f4119d295d"
format_stamp: "Formatted at 2026-06-19 12:04:38 on dist-test-slave-44vj"
server_key: "b83d590c19b28bab5cfeeca7e7101b5e"
server_key_iv: "0b75a766065df2080786f24fcc3a4ee5"
server_key_version: "encryptionkey@0"
I20260619 12:04:38.967252 2001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:38.991495 2001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:38.993901 2001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:38.994057 2001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:38.994596 2001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:38.994633 2001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:38.994668 2001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:38.994686 2001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:04:38 dist-test-slave-44vj krb5kdc[1884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:39.002775 2001 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:36115
I20260619 12:04:39.002893 2131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:36115 every 8 connection(s)
I20260619 12:04:39.003285 2001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:04:39.006266 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:38.995413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:41839 (local address 127.0.203.190:40467)
0619 12:04:38.995507 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:38.995510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:38.995726 (+ 216us) server_negotiation.cc:407] Connection header received
0619 12:04:38.996421 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:38.996423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:38.996439 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:38.996546 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:38.997688 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:38.998495 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:38.999789 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.000077 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.000139 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.002650 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.002652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.002652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.002664 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.004304 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.004818 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.004819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.004819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.004898 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.005473 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.005474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.005475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.005623 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.005703 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.006073 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.006192 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:04:39.006412 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:38.995322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40467 (local address 127.0.203.129:41839)
0619 12:04:38.995562 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:38.995614 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:38.996307 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:38.996650 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:38.996656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:38.997131 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:38.997567 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:38.997575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:38.998623 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:38.998624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:38.999692 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:38.999696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.000164 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.000165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.000299 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.000918 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:39.000928 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:39.002514 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:39.004387 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.004391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.004393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.004684 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.004966 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.004966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.004966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.005065 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.005729 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:39.005736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:39.005887 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.006008 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.006176 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":146,"threads_started":1}
I20260619 12:04:39.007213 2132 heartbeater.cc:347] Connected to a master server at 127.0.203.190:40467
I20260619 12:04:39.007315 2132 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:39.007576 2132 heartbeater.cc:511] Master 127.0.203.190:40467 requested a full tablet report, sending...
I20260619 12:04:39.008280 1928 ts_manager.cc:194] Registered new tserver with Master: c6d719b125444d189d4880f4119d295d (127.0.203.129:36115)
I20260619 12:04:39.009038 1928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:41839
I20260619 12:04:39.010807 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 2001
I20260619 12:04:39.010923 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:39.011116 814 external_mini_cluster.cc:1428] Setting key 921773263398a18176d4c68dcd3a3174
I20260619 12:04:39.014855 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.008521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:42161 (local address 127.0.203.190:40467)
0619 12:04:39.008564 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.008565 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.008736 (+ 171us) server_negotiation.cc:407] Connection header received
0619 12:04:39.008797 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.008798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.008808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.008865 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:39.009238 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.009944 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.010916 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.011225 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.011297 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.012348 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.012350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.012351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.012358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.013658 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.014024 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.014026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.014026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.014127 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.014329 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.014330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.014330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.014449 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.014504 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.014652 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.014763 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:04:39.014855 2138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.008466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40467 (local address 127.0.203.129:42161)
0619 12:04:39.008657 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.008670 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.008739 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.008872 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.008872 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.009069 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:39.009140 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.009142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.010090 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.010092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.010806 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.010809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.011303 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.011305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.011416 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.011906 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:39.011911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:39.012263 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:39.013733 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.013734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.013735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.013933 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.014212 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.014212 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.014213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.014276 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.014503 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:39.014504 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:39.014563 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.014625 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.014749 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:39.039188 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:40467
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:39.123976 2141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:39.124198 2141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:39.124222 2141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:39.125407 2141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:39.125458 2141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:39.125478 2141 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:04:39.125494 2141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:39.126981 2141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:40467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:39.127207 2141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:39.127465 2141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:39.132423 2147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:39.132385 2149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:39.132421 2146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:39.132987 2141 server_base.cc:1034] running on GCE node
I20260619 12:04:39.133229 2141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:39.133558 2141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:39.134739 2141 hybrid_clock.cc:648] HybridClock initialized: now 1781870679134720 us; error 39 us; skew 500 ppm
Jun 19 12:04:39 dist-test-slave-44vj krb5kdc[1884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:39.137524 2141 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:39.138181 2141 webserver.cc:460] Webserver started at http://127.0.203.130:43889/ using document root <none> and password file <none>
I20260619 12:04:39.138435 2141 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:39.138484 2141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:39.138664 2141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:39.139968 2141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "4e5e5b70542241aca388fac9403983e9"
format_stamp: "Formatted at 2026-06-19 12:04:39 on dist-test-slave-44vj"
server_key: "3504fcf97877f018771be72e267ddc43"
server_key_iv: "9614ebf3e96c78f7249af815fb2a4e70"
server_key_version: "encryptionkey@0"
I20260619 12:04:39.140359 2141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "4e5e5b70542241aca388fac9403983e9"
format_stamp: "Formatted at 2026-06-19 12:04:39 on dist-test-slave-44vj"
server_key: "3504fcf97877f018771be72e267ddc43"
server_key_iv: "9614ebf3e96c78f7249af815fb2a4e70"
server_key_version: "encryptionkey@0"
I20260619 12:04:39.142019 2141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:04:39.143059 2156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.143421 2141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:39.143597 2141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "4e5e5b70542241aca388fac9403983e9"
format_stamp: "Formatted at 2026-06-19 12:04:39 on dist-test-slave-44vj"
server_key: "3504fcf97877f018771be72e267ddc43"
server_key_iv: "9614ebf3e96c78f7249af815fb2a4e70"
server_key_version: "encryptionkey@0"
I20260619 12:04:39.143676 2141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:39.169971 2141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:39.172956 2141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:39.173149 2141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:39.173813 2141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:39.173856 2141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.173897 2141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:39.173920 2141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:04:39 dist-test-slave-44vj krb5kdc[1884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:39.182336 2141 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:38605
I20260619 12:04:39.182441 2271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:38605 every 8 connection(s)
I20260619 12:04:39.182840 2141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:39.185052 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 2141
I20260619 12:04:39.185148 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:39.185312 814 external_mini_cluster.cc:1428] Setting key 1f2ed6d3525dda325d31cd040c57f669
I20260619 12:04:39.185575 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.174568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:54753 (local address 127.0.203.190:40467)
0619 12:04:39.174630 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.174632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.174954 (+ 322us) server_negotiation.cc:407] Connection header received
0619 12:04:39.175636 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.175638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.175650 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.175709 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:39.176667 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.177521 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.179142 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.179514 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.179615 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.182165 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.182166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.182167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.182175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.183746 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.184258 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.184260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.184260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.184339 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.184727 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.184729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.184729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.184920 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.185028 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.185380 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.185494 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:04:39.185858 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.174542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40467 (local address 127.0.203.130:54753)
0619 12:04:39.174791 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.174851 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.175524 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.175793 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.175797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.176198 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:39.176546 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.176553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.177670 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.177671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.179025 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.179030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.179634 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.179635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.179784 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.180447 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:39.180463 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:39.182021 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:39.183838 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.183843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.183845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.184133 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.184439 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.184440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.184441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.184617 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.185019 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:39.185025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:39.185209 (+ 184us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.185320 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.185591 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
I20260619 12:04:39.186427 2272 heartbeater.cc:347] Connected to a master server at 127.0.203.190:40467
I20260619 12:04:39.186527 2272 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:39.186738 2272 heartbeater.cc:511] Master 127.0.203.190:40467 requested a full tablet report, sending...
I20260619 12:04:39.187537 1929 ts_manager.cc:194] Registered new tserver with Master: 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605)
I20260619 12:04:39.188112 1929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:54753
I20260619 12:04:39.194181 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.187889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:43919 (local address 127.0.203.190:40467)
0619 12:04:39.187925 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.187927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.187934 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:04:39.187985 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.187985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.187995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.188048 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:39.188408 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.188920 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.189873 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.190126 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.190197 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.191340 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.191342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.191342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.191349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.192866 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.193335 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.193336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.193336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.193400 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.193616 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.193616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.193617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.193772 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.193827 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.194032 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.194103 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:04:39.194252 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.187701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40467 (local address 127.0.203.130:43919)
0619 12:04:39.187854 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.187866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.187934 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.188056 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.188057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.188279 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:39.188351 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.188353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.189007 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.189008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.189746 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.189749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.190212 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.190213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.190286 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.190803 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:39.190809 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:39.191231 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:39.192968 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.192969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.192970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.193225 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.193482 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.193483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.193483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.193549 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.193840 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:39.193841 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:39.193910 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.194014 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.194138 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:39.215528 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:40467
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:39.297899 2281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:39.298137 2281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:39.298238 2281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:39.299855 2281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:39.299928 2281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:39.299954 2281 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:04:39.299973 2281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:39.302503 2281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:40467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:39.302791 2281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:39.303015 2281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:39.308722 2289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:39.308743 2286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:39.308749 2287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:39.308986 2281 server_base.cc:1034] running on GCE node
I20260619 12:04:39.309259 2281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:39.309549 2281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:39.310736 2281 hybrid_clock.cc:648] HybridClock initialized: now 1781870679310716 us; error 38 us; skew 500 ppm
Jun 19 12:04:39 dist-test-slave-44vj krb5kdc[1884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:39.313882 2281 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:39.314484 2281 webserver.cc:460] Webserver started at http://127.0.203.131:37465/ using document root <none> and password file <none>
I20260619 12:04:39.314708 2281 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:39.314765 2281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:39.314873 2281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:39.315995 2281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "cd4c5a80173f4268952654008e3278d8"
format_stamp: "Formatted at 2026-06-19 12:04:39 on dist-test-slave-44vj"
server_key: "1dc208d32f1a8bb5c3b641767ce29135"
server_key_iv: "b47f9230cea35e8efd51928d2cd9dbce"
server_key_version: "encryptionkey@0"
I20260619 12:04:39.316325 2281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "cd4c5a80173f4268952654008e3278d8"
format_stamp: "Formatted at 2026-06-19 12:04:39 on dist-test-slave-44vj"
server_key: "1dc208d32f1a8bb5c3b641767ce29135"
server_key_iv: "b47f9230cea35e8efd51928d2cd9dbce"
server_key_version: "encryptionkey@0"
I20260619 12:04:39.317804 2281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:39.318748 2296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.319013 2281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:39.319116 2281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "cd4c5a80173f4268952654008e3278d8"
format_stamp: "Formatted at 2026-06-19 12:04:39 on dist-test-slave-44vj"
server_key: "1dc208d32f1a8bb5c3b641767ce29135"
server_key_iv: "b47f9230cea35e8efd51928d2cd9dbce"
server_key_version: "encryptionkey@0"
I20260619 12:04:39.319226 2281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:39.327229 2281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:39.329788 2281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:39.329923 2281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:39.330591 2281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:39.330637 2281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.330667 2281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:39.330680 2281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.338490 2281 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:33591
Jun 19 12:04:39 dist-test-slave-44vj krb5kdc[1884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:39.338579 2411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:33591 every 8 connection(s)
I20260619 12:04:39.338965 2281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:04:39.340835 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 2281
I20260619 12:04:39.340929 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:39.341142 814 external_mini_cluster.cc:1428] Setting key 37e822f90530a19fe99c6b5c56c8bb1f
I20260619 12:04:39.342383 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.331268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:34349 (local address 127.0.203.190:40467)
0619 12:04:39.331320 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.331322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.331634 (+ 312us) server_negotiation.cc:407] Connection header received
0619 12:04:39.332217 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.332218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.332232 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.332315 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:39.333361 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.334088 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.335785 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.336088 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.336161 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.339172 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.339174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.339175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.339184 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.340694 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.341212 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.341214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.341214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.341281 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.341609 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.341611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.341611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.341802 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.341880 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.342224 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.342312 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:04:39.342602 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.331277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40467 (local address 127.0.203.131:34349)
0619 12:04:39.331490 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.331533 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.332116 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.332468 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.332476 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.332876 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:39.333237 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.333244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.334201 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.334203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.335681 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.335685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.336211 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.336213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.336412 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.337035 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:39.337047 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:39.339058 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:39.340795 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.340802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.340805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.341119 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.341354 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.341356 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.341356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.341509 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.341878 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:39.341883 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:39.342074 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.342197 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.342370 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260619 12:04:39.343127 2412 heartbeater.cc:347] Connected to a master server at 127.0.203.190:40467
I20260619 12:04:39.343215 2412 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:39.343472 2412 heartbeater.cc:511] Master 127.0.203.190:40467 requested a full tablet report, sending...
I20260619 12:04:39.343917 1928 ts_manager.cc:194] Registered new tserver with Master: cd4c5a80173f4268952654008e3278d8 (127.0.203.131:33591)
I20260619 12:04:39.344323 1928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:34349
I20260619 12:04:39.350551 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.344406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:60593 (local address 127.0.203.190:40467)
0619 12:04:39.344507 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.344508 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.344585 (+ 77us) server_negotiation.cc:407] Connection header received
0619 12:04:39.344642 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.344643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.344654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.344711 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:39.345103 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.345603 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.346311 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.346496 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.346555 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.347739 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.347740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.347740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.347747 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.349254 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.349683 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.349684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.349685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.349745 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.349970 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.349971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.349972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.350131 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.350204 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.350397 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.350470 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 12:04:39.350558 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.344341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40467 (local address 127.0.203.131:60593)
0619 12:04:39.344523 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.344534 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.344595 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.344719 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.344720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.344928 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:39.344991 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.344994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.345706 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.345706 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.346237 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.346239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.346561 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.346562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.346611 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.347250 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:39.347257 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:39.347619 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:39.349365 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.349366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.349367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.349571 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.349814 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.349814 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.349815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.349878 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.350233 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:39.350234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:39.350293 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.350368 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.350469 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260619 12:04:39.353091 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:04:39 dist-test-slave-44vj krb5kdc[1884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:04:39 dist-test-slave-44vj krb5kdc[1884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:39.390527 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.381965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.0.203.190:40467)
0619 12:04:39.382077 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.382080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.382094 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:04:39.382136 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.382138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.382156 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.382271 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:39.383129 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.383969 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.384809 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.385026 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.385085 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.388106 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.388107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.388108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.388119 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.389421 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.389862 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.389863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.389863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.389918 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.390073 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.390073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.390074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.390187 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.390271 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.390380 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.390446 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 12:04:39.392576 1928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:39.394505 1928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:39.402511 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.397750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.0.203.129:36115)
0619 12:04:39.398600 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.398606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.398618 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:04:39.398665 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.398667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.398686 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.398774 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:39.399142 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.400340 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.401401 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.402069 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.402137 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.402232 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.402347 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.402392 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":123,"threads_started":1}
I20260619 12:04:39.402621 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.398081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:38605 (local address 127.0.0.1:44884)
0619 12:04:39.398354 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.398365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.398454 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.398848 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.398849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.398853 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:39.398929 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.398932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.400014 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.400016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.401277 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.401280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.402308 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.402309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.402388 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.402395 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.402427 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.402487 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":5,"thread_start_us":108,"threads_started":1}
I20260619 12:04:39.402500 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.397641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:36115 (local address 127.0.0.1:42004)
0619 12:04:39.397959 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.398011 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.398115 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.399014 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.399014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.399016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:39.399049 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.399050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.400441 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.400442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.401311 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.401313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.402149 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.402150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.402237 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.402252 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.402330 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.402400 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":44,"thread_start_us":63,"threads_started":1}
I20260619 12:04:39.402844 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.398369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.0.203.130:38605)
0619 12:04:39.398684 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.398690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.398701 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:04:39.398740 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.398742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.398758 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.398849 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:39.399001 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.399937 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.401470 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.402204 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.402581 (+ 377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.402653 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.402684 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.402728 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":90,"threads_started":1}
I20260619 12:04:39.403271 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.397707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:33591 (local address 127.0.0.1:44700)
0619 12:04:39.398253 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.398266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.398378 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.398742 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.398746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.398748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:39.398889 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.398893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.400439 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.400440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.401629 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.401631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.403043 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.403045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.403106 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.403111 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.403153 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.403202 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":44,"thread_start_us":158,"threads_started":1}
I20260619 12:04:39.403559 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.397901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44700 (local address 127.0.203.131:33591)
0619 12:04:39.398071 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.398077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.398341 (+ 264us) server_negotiation.cc:407] Connection header received
0619 12:04:39.398470 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.398473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.398561 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.398630 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:39.399013 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.400342 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.401732 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.402470 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.402526 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.402767 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.403418 (+ 651us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.403464 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260619 12:04:39.403731 2206 tablet_service.cc:1467] Processing CreateTablet for tablet 1170460ac2384e1b90c4ba47b3f7b34f (DEFAULT_TABLE table=test-table [id=2b393990f35841a1b6b18e967ef93873]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:39.403731 2061 tablet_service.cc:1467] Processing CreateTablet for tablet 1170460ac2384e1b90c4ba47b3f7b34f (DEFAULT_TABLE table=test-table [id=2b393990f35841a1b6b18e967ef93873]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:39.404176 2061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1170460ac2384e1b90c4ba47b3f7b34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.404176 2206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1170460ac2384e1b90c4ba47b3f7b34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.404218 2346 tablet_service.cc:1467] Processing CreateTablet for tablet 1170460ac2384e1b90c4ba47b3f7b34f (DEFAULT_TABLE table=test-table [id=2b393990f35841a1b6b18e967ef93873]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:39.404601 2346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1170460ac2384e1b90c4ba47b3f7b34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.408422 2435 tablet_bootstrap.cc:492] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Bootstrap starting.
I20260619 12:04:39.409179 2435 tablet_bootstrap.cc:654] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.409472 2435 log.cc:824] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:39.409854 2436 tablet_bootstrap.cc:492] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Bootstrap starting.
I20260619 12:04:39.410323 2435 tablet_bootstrap.cc:492] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: No bootstrap required, opened a new log
I20260619 12:04:39.410392 2436 tablet_bootstrap.cc:654] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.410413 2435 ts_tablet_manager.cc:1397] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:04:39.410655 2436 log.cc:824] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:39.411015 2437 tablet_bootstrap.cc:492] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Bootstrap starting.
I20260619 12:04:39.411712 2436 tablet_bootstrap.cc:492] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: No bootstrap required, opened a new log
I20260619 12:04:39.411809 2437 tablet_bootstrap.cc:654] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.411906 2436 ts_tablet_manager.cc:1397] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:04:39.412184 2437 log.cc:824] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:39.412923 2437 tablet_bootstrap.cc:492] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: No bootstrap required, opened a new log
I20260619 12:04:39.412779 2435 raft_consensus.cc:348] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.412983 2435 raft_consensus.cc:374] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.412997 2437 ts_tablet_manager.cc:1397] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:04:39.413031 2435 raft_consensus.cc:729] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4c5a80173f4268952654008e3278d8, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.413183 2435 consensus_queue.cc:260] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.413520 2435 ts_tablet_manager.cc:1428] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:39.413574 2412 heartbeater.cc:503] Master 127.0.203.190:40467 was elected leader, sending a full tablet report...
I20260619 12:04:39.414147 2436 raft_consensus.cc:348] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.414286 2436 raft_consensus.cc:374] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.414317 2436 raft_consensus.cc:729] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d719b125444d189d4880f4119d295d, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.414463 2436 consensus_queue.cc:260] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.414778 2132 heartbeater.cc:503] Master 127.0.203.190:40467 was elected leader, sending a full tablet report...
I20260619 12:04:39.414762 2436 ts_tablet_manager.cc:1428] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:39.415472 2437 raft_consensus.cc:348] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.415720 2437 raft_consensus.cc:374] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.415769 2437 raft_consensus.cc:729] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5e5b70542241aca388fac9403983e9, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.415912 2437 consensus_queue.cc:260] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.416256 2437 ts_tablet_manager.cc:1428] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:39.416306 2272 heartbeater.cc:503] Master 127.0.203.190:40467 was elected leader, sending a full tablet report...
W20260619 12:04:39.433748 2273 tablet.cc:2369] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:04:39.504207 2133 tablet.cc:2369] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:39.586016 2442 raft_consensus.cc:482] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:39.586194 2442 raft_consensus.cc:504] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.586647 2442 leader_election.cc:288] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4c5a80173f4268952654008e3278d8 (127.0.203.131:33591), 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605)
W20260619 12:04:39.589736 2413 tablet.cc:2369] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:39.590209 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.587048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:49133 (local address 127.0.203.131:33591)
0619 12:04:39.587179 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.587181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.587278 (+ 97us) server_negotiation.cc:407] Connection header received
0619 12:04:39.587395 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.587396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.587405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.587463 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:39.587634 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.588231 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.589386 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.589799 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.589873 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.589884 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.590083 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.590124 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260619 12:04:39.590361 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.587026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:56935 (local address 127.0.203.130:38605)
0619 12:04:39.587082 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.587084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.587196 (+ 112us) server_negotiation.cc:407] Connection header received
0619 12:04:39.587307 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.587308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.587319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.587426 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:39.587613 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.588146 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.589466 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.589962 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.590042 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.590052 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.590244 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.590289 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:04:39.590358 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.586902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:33591 (local address 127.0.203.129:49133)
0619 12:04:39.587196 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.587209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.587289 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.587476 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.587477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.587480 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:39.587559 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.587561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.588328 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.588329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.589257 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.589264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.589928 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.589929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.590008 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.590014 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.590067 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.590125 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":178,"thread_start_us":57,"threads_started":1}
I20260619 12:04:39.590374 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.586902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:38605 (local address 127.0.203.129:56935)
0619 12:04:39.587123 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.587134 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.587217 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.587442 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.587443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.587446 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:39.587534 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.587537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.588262 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.588264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.589349 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.589354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.590091 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.590092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.590172 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.590176 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.590222 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.590282 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":64,"thread_start_us":65,"threads_started":1}
I20260619 12:04:39.590785 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1170460ac2384e1b90c4ba47b3f7b34f" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4c5a80173f4268952654008e3278d8" is_pre_election: true
I20260619 12:04:39.591069 2366 raft_consensus.cc:2454] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 0.
I20260619 12:04:39.591038 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1170460ac2384e1b90c4ba47b3f7b34f" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5e5b70542241aca388fac9403983e9" is_pre_election: true
I20260619 12:04:39.591178 2226 raft_consensus.cc:2454] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 0.
I20260619 12:04:39.591409 2018 leader_election.cc:302] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6d719b125444d189d4880f4119d295d, cd4c5a80173f4268952654008e3278d8; no voters:
I20260619 12:04:39.591599 2442 raft_consensus.cc:2781] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:39.591667 2442 raft_consensus.cc:482] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:39.591684 2442 raft_consensus.cc:3037] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:39.592517 2442 raft_consensus.cc:504] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.592684 2442 leader_election.cc:288] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 election: Requested vote from peers cd4c5a80173f4268952654008e3278d8 (127.0.203.131:33591), 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605)
I20260619 12:04:39.592869 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1170460ac2384e1b90c4ba47b3f7b34f" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4c5a80173f4268952654008e3278d8"
I20260619 12:04:39.592913 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1170460ac2384e1b90c4ba47b3f7b34f" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5e5b70542241aca388fac9403983e9"
I20260619 12:04:39.592979 2366 raft_consensus.cc:3037] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:39.592989 2226 raft_consensus.cc:3037] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:39.593822 2226 raft_consensus.cc:2454] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 1.
I20260619 12:04:39.593878 2366 raft_consensus.cc:2454] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 1.
I20260619 12:04:39.594043 2019 leader_election.cc:302] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5e5b70542241aca388fac9403983e9, c6d719b125444d189d4880f4119d295d; no voters:
I20260619 12:04:39.594221 2442 raft_consensus.cc:2781] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:39.594375 2442 raft_consensus.cc:686] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 1 LEADER]: Becoming Leader. State: Replica: c6d719b125444d189d4880f4119d295d, State: Running, Role: LEADER
I20260619 12:04:39.594494 2442 consensus_queue.cc:237] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } }
I20260619 12:04:39.595476 1928 catalog_manager.cc:5495] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d reported cstate change: term changed from 0 to 1, leader changed from <none> to c6d719b125444d189d4880f4119d295d (127.0.203.129). New cstate: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } health_report { overall_health: UNKNOWN } } }
I20260619 12:04:39.638636 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.636018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.0.203.129:36115)
0619 12:04:39.636090 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.636092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.636102 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:04:39.636188 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.636189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.636211 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.636262 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:39.636429 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.636991 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.637837 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.638024 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.638081 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.638293 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:39.638399 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:39.638478 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.638530 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.638551 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:04:39.640976 2366 raft_consensus.cc:1261] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Refusing update from remote peer c6d719b125444d189d4880f4119d295d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:39.640972 2226 raft_consensus.cc:1261] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Refusing update from remote peer c6d719b125444d189d4880f4119d295d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:39.641413 2446 consensus_queue.cc:1035] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:39.641566 2442 consensus_queue.cc:1035] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:39.644769 2449 mvcc.cc:205] Tried to move back new op lower bound from 7298542303805743104 to 7298542303619608576. Current Snapshot: MvccSnapshot[applied={T|T < 7298542303805743104}]
I20260619 12:04:39.644826 2450 mvcc.cc:205] Tried to move back new op lower bound from 7298542303805743104 to 7298542303619608576. Current Snapshot: MvccSnapshot[applied={T|T < 7298542303805743104}]
I20260619 12:04:39.648964 1928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:39.649101 1928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:39.649946 1928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223: Sending DeleteTablet for 3 replicas of tablet 1170460ac2384e1b90c4ba47b3f7b34f
I20260619 12:04:39.650322 2346 tablet_service.cc:1514] Processing DeleteTablet for tablet 1170460ac2384e1b90c4ba47b3f7b34f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:39 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:44700
I20260619 12:04:39.650401 2061 tablet_service.cc:1514] Processing DeleteTablet for tablet 1170460ac2384e1b90c4ba47b3f7b34f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:39 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42004
I20260619 12:04:39.650393 2206 tablet_service.cc:1514] Processing DeleteTablet for tablet 1170460ac2384e1b90c4ba47b3f7b34f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:39 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:44884
I20260619 12:04:39.650754 2461 tablet_replica.cc:330] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: stopping tablet replica
I20260619 12:04:39.650983 1928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 12:04:39.651041 2459 tablet_replica.cc:330] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: stopping tablet replica
W20260619 12:04:39.651137 1928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:39.651172 2459 raft_consensus.cc:2229] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:39.651342 2459 raft_consensus.cc:2258] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:39.650905 2461 raft_consensus.cc:2229] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:04:39.651822 2461 raft_consensus.cc:2258] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:39.651927 2459 ts_tablet_manager.cc:1905] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:39.652026 2460 tablet_replica.cc:330] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: stopping tablet replica
I20260619 12:04:39.652153 2460 raft_consensus.cc:2229] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:39.652273 2460 raft_consensus.cc:2258] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:39.652767 2346 tablet_service.cc:1467] Processing CreateTablet for tablet 0ded7dd6c7a446fdb510efaf92dc92d9 (DEFAULT_TABLE table=test-table [id=e3d2c9814d97441f8ad0a35c5f55f6cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:39.652799 2461 ts_tablet_manager.cc:1905] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:39.652988 2346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ded7dd6c7a446fdb510efaf92dc92d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.653831 2459 ts_tablet_manager.cc:1918] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:04:39.653910 2459 log.cc:1196] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/1170460ac2384e1b90c4ba47b3f7b34f
I20260619 12:04:39.654204 2459 ts_tablet_manager.cc:1939] T 1170460ac2384e1b90c4ba47b3f7b34f P cd4c5a80173f4268952654008e3278d8: Deleting consensus metadata
I20260619 12:04:39.654886 2435 tablet_bootstrap.cc:492] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: Bootstrap starting.
I20260619 12:04:39.654942 2461 ts_tablet_manager.cc:1918] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:04:39.655015 2461 log.cc:1196] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/1170460ac2384e1b90c4ba47b3f7b34f
I20260619 12:04:39.655015 1914 catalog_manager.cc:4841] TS cd4c5a80173f4268952654008e3278d8 (127.0.203.131:33591): tablet 1170460ac2384e1b90c4ba47b3f7b34f (table test-table [id=2b393990f35841a1b6b18e967ef93873]) successfully deleted
I20260619 12:04:39.655328 2435 tablet_bootstrap.cc:654] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.655385 2461 ts_tablet_manager.cc:1939] T 1170460ac2384e1b90c4ba47b3f7b34f P c6d719b125444d189d4880f4119d295d: Deleting consensus metadata
I20260619 12:04:39.655958 2206 tablet_service.cc:1467] Processing CreateTablet for tablet 0ded7dd6c7a446fdb510efaf92dc92d9 (DEFAULT_TABLE table=test-table [id=e3d2c9814d97441f8ad0a35c5f55f6cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:39.656068 2435 tablet_bootstrap.cc:492] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: No bootstrap required, opened a new log
I20260619 12:04:39.656102 2206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ded7dd6c7a446fdb510efaf92dc92d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.656131 2435 ts_tablet_manager.cc:1397] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:39.656322 2435 raft_consensus.cc:348] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.656430 2435 raft_consensus.cc:374] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.656463 2435 raft_consensus.cc:729] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4c5a80173f4268952654008e3278d8, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.656515 2435 consensus_queue.cc:260] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.656620 2435 ts_tablet_manager.cc:1428] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 12:04:39.657621 2437 tablet_bootstrap.cc:492] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: Bootstrap starting.
I20260619 12:04:39.657842 1916 catalog_manager.cc:4841] TS c6d719b125444d189d4880f4119d295d (127.0.203.129:36115): tablet 1170460ac2384e1b90c4ba47b3f7b34f (table test-table [id=2b393990f35841a1b6b18e967ef93873]) successfully deleted
I20260619 12:04:39.657927 2061 tablet_service.cc:1467] Processing CreateTablet for tablet 0ded7dd6c7a446fdb510efaf92dc92d9 (DEFAULT_TABLE table=test-table [id=e3d2c9814d97441f8ad0a35c5f55f6cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:39.658025 2061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ded7dd6c7a446fdb510efaf92dc92d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.658139 2437 tablet_bootstrap.cc:654] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.658974 2460 ts_tablet_manager.cc:1905] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:39.659539 2436 tablet_bootstrap.cc:492] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: Bootstrap starting.
I20260619 12:04:39.659847 2437 tablet_bootstrap.cc:492] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: No bootstrap required, opened a new log
I20260619 12:04:39.659901 2437 ts_tablet_manager.cc:1397] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:04:39.660015 2436 tablet_bootstrap.cc:654] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.660074 2437 raft_consensus.cc:348] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.660161 2437 raft_consensus.cc:374] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.660179 2437 raft_consensus.cc:729] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5e5b70542241aca388fac9403983e9, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.660234 2437 consensus_queue.cc:260] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.660326 2437 ts_tablet_manager.cc:1428] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:04:39.661458 2436 tablet_bootstrap.cc:492] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: No bootstrap required, opened a new log
I20260619 12:04:39.661511 2436 ts_tablet_manager.cc:1397] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:04:39.661684 2436 raft_consensus.cc:348] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.661773 2436 raft_consensus.cc:374] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.661792 2436 raft_consensus.cc:729] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d719b125444d189d4880f4119d295d, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.661831 2436 consensus_queue.cc:260] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.661934 2436 ts_tablet_manager.cc:1428] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.662634 2460 ts_tablet_manager.cc:1918] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:04:39.662834 2460 log.cc:1196] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/1170460ac2384e1b90c4ba47b3f7b34f
I20260619 12:04:39.663138 2460 ts_tablet_manager.cc:1939] T 1170460ac2384e1b90c4ba47b3f7b34f P 4e5e5b70542241aca388fac9403983e9: Deleting consensus metadata
I20260619 12:04:39.663753 1915 catalog_manager.cc:4841] TS 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605): tablet 1170460ac2384e1b90c4ba47b3f7b34f (table test-table [id=2b393990f35841a1b6b18e967ef93873]) successfully deleted
I20260619 12:04:39.880827 2441 raft_consensus.cc:482] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:39.880985 2441 raft_consensus.cc:504] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.881412 2441 leader_election.cc:288] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6d719b125444d189d4880f4119d295d (127.0.203.129:36115), 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605)
I20260619 12:04:39.884913 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.881808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:54219 (local address 127.0.203.129:36115)
0619 12:04:39.881894 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.881897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.882058 (+ 161us) server_negotiation.cc:407] Connection header received
0619 12:04:39.882136 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.882137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.882150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.882203 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:39.882407 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.882915 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.884155 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.884515 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.884580 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.884590 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.884778 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.884826 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:04:39.884917 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.881891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:47631 (local address 127.0.203.130:38605)
0619 12:04:39.881974 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.881977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.882045 (+ 68us) server_negotiation.cc:407] Connection header received
0619 12:04:39.882164 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.882165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.882175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.882237 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:39.882441 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.882961 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.884166 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.884539 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.884612 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.884624 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.884770 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.884822 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:04:39.884972 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.881672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:36115 (local address 127.0.203.131:54219)
0619 12:04:39.881916 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.881932 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.882062 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.882226 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.882227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.882231 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:39.882337 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.882340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.882997 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.882998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.884050 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.884053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.884600 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.884602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.884695 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.884701 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.884754 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.884824 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":91,"thread_start_us":75,"threads_started":1}
I20260619 12:04:39.884997 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.881673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:38605 (local address 127.0.203.131:47631)
0619 12:04:39.881972 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.881983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.882063 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.882263 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.882264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.882268 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:39.882357 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.882360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.883056 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.883058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.884048 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.884052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.884628 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.884630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.884690 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.884694 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.884746 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.884819 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":5,"thread_start_us":74,"threads_started":1}
I20260619 12:04:39.885288 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ded7dd6c7a446fdb510efaf92dc92d9" candidate_uuid: "cd4c5a80173f4268952654008e3278d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5e5b70542241aca388fac9403983e9" is_pre_election: true
I20260619 12:04:39.885314 2086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ded7dd6c7a446fdb510efaf92dc92d9" candidate_uuid: "cd4c5a80173f4268952654008e3278d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d719b125444d189d4880f4119d295d" is_pre_election: true
I20260619 12:04:39.885413 2226 raft_consensus.cc:2454] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4c5a80173f4268952654008e3278d8 in term 0.
I20260619 12:04:39.885465 2086 raft_consensus.cc:2454] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4c5a80173f4268952654008e3278d8 in term 0.
I20260619 12:04:39.885629 2299 leader_election.cc:302] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5e5b70542241aca388fac9403983e9, cd4c5a80173f4268952654008e3278d8; no voters:
I20260619 12:04:39.885850 2441 raft_consensus.cc:2781] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:39.885898 2441 raft_consensus.cc:482] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:39.885927 2441 raft_consensus.cc:3037] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:39.886837 2441 raft_consensus.cc:504] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.887037 2441 leader_election.cc:288] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [CANDIDATE]: Term 1 election: Requested vote from peers c6d719b125444d189d4880f4119d295d (127.0.203.129:36115), 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605)
I20260619 12:04:39.887238 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ded7dd6c7a446fdb510efaf92dc92d9" candidate_uuid: "cd4c5a80173f4268952654008e3278d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5e5b70542241aca388fac9403983e9"
I20260619 12:04:39.887351 2226 raft_consensus.cc:3037] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:39.888262 2226 raft_consensus.cc:2454] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4c5a80173f4268952654008e3278d8 in term 1.
I20260619 12:04:39.888415 2086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ded7dd6c7a446fdb510efaf92dc92d9" candidate_uuid: "cd4c5a80173f4268952654008e3278d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d719b125444d189d4880f4119d295d"
I20260619 12:04:39.888466 2299 leader_election.cc:302] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5e5b70542241aca388fac9403983e9, cd4c5a80173f4268952654008e3278d8; no voters:
I20260619 12:04:39.888517 2086 raft_consensus.cc:3037] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:39.888578 2441 raft_consensus.cc:2781] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:39.888722 2441 raft_consensus.cc:686] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 1 LEADER]: Becoming Leader. State: Replica: cd4c5a80173f4268952654008e3278d8, State: Running, Role: LEADER
I20260619 12:04:39.888805 2441 consensus_queue.cc:237] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.889593 2086 raft_consensus.cc:2454] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4c5a80173f4268952654008e3278d8 in term 1.
I20260619 12:04:39.889740 1928 catalog_manager.cc:5495] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd4c5a80173f4268952654008e3278d8 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "cd4c5a80173f4268952654008e3278d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } health_report { overall_health: HEALTHY } } }
Jun 19 12:04:39 dist-test-slave-44vj krb5kdc[1884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:39.968405 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.960409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40467 (local address 127.0.0.1:41090)
0619 12:04:39.960772 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:39.960785 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:39.960886 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:39.961046 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:39.961047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:39.961436 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:39.961496 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.961498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.962207 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.962208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.962804 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:39.962806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.963133 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.963134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:39.963179 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.963812 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:39.963818 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:39.965385 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:39.967112 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.967118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.967121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.967338 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.967658 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:39.967659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:39.967659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:39.967721 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:39.968020 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:39.968026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:39.968086 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 12:04:39.968174 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:04:39.968311 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":200,"threads_started":1}
I20260619 12:04:39.968497 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:39.960469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.0.203.190:40467)
0619 12:04:39.960709 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:39.960711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:39.960873 (+ 162us) server_negotiation.cc:407] Connection header received
0619 12:04:39.960950 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:39.960950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:39.960963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:39.961036 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:39.961588 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.962128 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.962872 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:39.963074 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:39.963130 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:39.965512 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:39.965514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:39.965514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:39.965522 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:39.967007 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.967478 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.967480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.967480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.967581 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:39.967776 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:39.967776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:39.967777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:39.967922 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:39.967989 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:39.968243 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 12:04:39.968383 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":200,"thread_start_us":149,"threads_started":1}
I20260619 12:04:39.971128 1928 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:41090:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:04:39.971423 1928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:39.973011 2346 tablet_service.cc:1467] Processing CreateTablet for tablet 5e2d6408d35a4ed093ee74b2fc204128 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd3909d7bd2742e5b1df8c088eaf2ace]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:04:39.973011 2061 tablet_service.cc:1467] Processing CreateTablet for tablet 5e2d6408d35a4ed093ee74b2fc204128 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd3909d7bd2742e5b1df8c088eaf2ace]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:04:39.973141 2206 tablet_service.cc:1467] Processing CreateTablet for tablet 5e2d6408d35a4ed093ee74b2fc204128 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd3909d7bd2742e5b1df8c088eaf2ace]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:04:39.973217 2346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e2d6408d35a4ed093ee74b2fc204128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.973217 2061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e2d6408d35a4ed093ee74b2fc204128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.973320 2206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e2d6408d35a4ed093ee74b2fc204128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:39.974582 2437 tablet_bootstrap.cc:492] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: Bootstrap starting.
I20260619 12:04:39.974939 2437 tablet_bootstrap.cc:654] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.974997 2435 tablet_bootstrap.cc:492] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: Bootstrap starting.
I20260619 12:04:39.975009 2436 tablet_bootstrap.cc:492] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: Bootstrap starting.
I20260619 12:04:39.975422 2436 tablet_bootstrap.cc:654] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.975651 2435 tablet_bootstrap.cc:654] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:39.975979 2437 tablet_bootstrap.cc:492] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: No bootstrap required, opened a new log
I20260619 12:04:39.976028 2437 ts_tablet_manager.cc:1397] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:39.976207 2436 tablet_bootstrap.cc:492] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: No bootstrap required, opened a new log
I20260619 12:04:39.976173 2437 raft_consensus.cc:348] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.976243 2436 ts_tablet_manager.cc:1397] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:39.976245 2437 raft_consensus.cc:374] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.976279 2437 raft_consensus.cc:729] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5e5b70542241aca388fac9403983e9, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.976323 2435 tablet_bootstrap.cc:492] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: No bootstrap required, opened a new log
I20260619 12:04:39.976323 2437 consensus_queue.cc:260] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.976361 2435 ts_tablet_manager.cc:1397] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:39.976354 2436 raft_consensus.cc:348] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.976429 2436 raft_consensus.cc:374] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.976435 2437 ts_tablet_manager.cc:1428] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.976452 2436 raft_consensus.cc:729] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d719b125444d189d4880f4119d295d, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.976495 2436 consensus_queue.cc:260] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.976517 2435 raft_consensus.cc:348] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.976588 2436 ts_tablet_manager.cc:1428] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.976542 2443 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:39.976615 2435 raft_consensus.cc:374] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:39.976637 2435 raft_consensus.cc:729] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4c5a80173f4268952654008e3278d8, State: Initialized, Role: FOLLOWER
I20260619 12:04:39.976641 2443 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:39.976686 2435 consensus_queue.cc:260] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:39.976780 2435 ts_tablet_manager.cc:1428] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:39.976866 2442 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:39.976863 2441 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:39.976954 2442 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:39.976986 2441 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:40.011205 2442 raft_consensus.cc:482] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:40.011397 2442 raft_consensus.cc:504] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:40.011621 2442 leader_election.cc:288] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605), cd4c5a80173f4268952654008e3278d8 (127.0.203.131:33591)
I20260619 12:04:40.011931 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e2d6408d35a4ed093ee74b2fc204128" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4c5a80173f4268952654008e3278d8" is_pre_election: true
I20260619 12:04:40.011931 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e2d6408d35a4ed093ee74b2fc204128" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5e5b70542241aca388fac9403983e9" is_pre_election: true
I20260619 12:04:40.012050 2366 raft_consensus.cc:2454] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 0.
I20260619 12:04:40.012050 2226 raft_consensus.cc:2454] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 0.
I20260619 12:04:40.012228 2019 leader_election.cc:302] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5e5b70542241aca388fac9403983e9, c6d719b125444d189d4880f4119d295d; no voters:
I20260619 12:04:40.012408 2442 raft_consensus.cc:2781] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:40.012471 2442 raft_consensus.cc:482] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:40.012492 2442 raft_consensus.cc:3037] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:40.013324 2442 raft_consensus.cc:504] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:40.013473 2442 leader_election.cc:288] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 election: Requested vote from peers 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605), cd4c5a80173f4268952654008e3278d8 (127.0.203.131:33591)
I20260619 12:04:40.013716 2226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e2d6408d35a4ed093ee74b2fc204128" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5e5b70542241aca388fac9403983e9"
I20260619 12:04:40.013801 2226 raft_consensus.cc:3037] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:40.013772 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e2d6408d35a4ed093ee74b2fc204128" candidate_uuid: "c6d719b125444d189d4880f4119d295d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4c5a80173f4268952654008e3278d8"
I20260619 12:04:40.013911 2366 raft_consensus.cc:3037] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:40.014562 2226 raft_consensus.cc:2454] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 1.
I20260619 12:04:40.014830 2019 leader_election.cc:302] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5e5b70542241aca388fac9403983e9, c6d719b125444d189d4880f4119d295d; no voters:
I20260619 12:04:40.014847 2366 raft_consensus.cc:2454] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6d719b125444d189d4880f4119d295d in term 1.
I20260619 12:04:40.014961 2442 raft_consensus.cc:2781] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:40.015058 2442 raft_consensus.cc:686] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [term 1 LEADER]: Becoming Leader. State: Replica: c6d719b125444d189d4880f4119d295d, State: Running, Role: LEADER
I20260619 12:04:40.015120 2442 consensus_queue.cc:237] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } }
I20260619 12:04:40.015533 2446 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: TxnStatusTablet state changed. Reason: New leader c6d719b125444d189d4880f4119d295d. Latest consensus state: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:40.015657 2446 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:04:40.015919 2480 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:04:40.015911 1929 catalog_manager.cc:5495] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d reported cstate change: term changed from 0 to 1, leader changed from <none> to c6d719b125444d189d4880f4119d295d (127.0.203.129). New cstate: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } health_report { overall_health: UNKNOWN } } }
I20260619 12:04:40.016069 2480 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:04:40.029189 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.026658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:36115 (local address 127.0.0.1:42018)
0619 12:04:40.026864 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:40.026875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:40.026936 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:40.027078 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:40.027079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:40.027080 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:40.027142 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:40.027145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.027826 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.027827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:40.028417 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:40.028419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.028757 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.028758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:40.028823 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.028850 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:40.029011 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:40.029019 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:04:40.029061 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 12:04:40.029118 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260619 12:04:40.029175 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.026769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.0.203.129:36115)
0619 12:04:40.026828 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:40.026831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:40.026930 (+ 99us) server_negotiation.cc:407] Connection header received
0619 12:04:40.027005 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:40.027006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:40.027018 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:40.027064 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:40.027201 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.027740 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.028507 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.028693 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.028752 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.028922 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:40.028955 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:40.029004 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:40.029063 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:04:40.029087 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:04:40.030505 2366 raft_consensus.cc:1261] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Refusing update from remote peer c6d719b125444d189d4880f4119d295d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:40.030570 2226 raft_consensus.cc:1261] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Refusing update from remote peer c6d719b125444d189d4880f4119d295d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:40.030758 2446 consensus_queue.cc:1035] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:40.030872 2442 consensus_queue.cc:1035] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:40.031638 2441 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: TxnStatusTablet state changed. Reason: New leader c6d719b125444d189d4880f4119d295d. Latest consensus state: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:40.031736 2441 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:40.032106 2443 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: TxnStatusTablet state changed. Reason: New leader c6d719b125444d189d4880f4119d295d. Latest consensus state: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:40.032402 2443 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:40.033120 2446 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:40.033237 2446 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:04:40.033582 2454 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:40.033666 2454 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P c6d719b125444d189d4880f4119d295d: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:04:40.034118 2451 mvcc.cc:205] Tried to move back new op lower bound from 7298542305402646528 to 7298542305342652416. Current Snapshot: MvccSnapshot[applied={T|T < 7298542305402646528}]
I20260619 12:04:40.034273 2443 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:40.034359 2443 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P 4e5e5b70542241aca388fac9403983e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:40.035537 2441 tablet_replica.cc:439] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6d719b125444d189d4880f4119d295d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 } } peers { permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 } } peers { permanent_uuid: "cd4c5a80173f4268952654008e3278d8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33591 } } }
I20260619 12:04:40.035666 2441 tablet_replica.cc:442] T 5e2d6408d35a4ed093ee74b2fc204128 P cd4c5a80173f4268952654008e3278d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:40.039431 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.035650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.0.203.131:33591)
0619 12:04:40.035692 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:40.035694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:40.035703 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:04:40.035788 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:40.035789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:40.035811 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:40.035966 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:40.036128 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.036676 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.038425 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.038641 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.038724 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.038917 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:40.039049 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:40.039111 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:40.039248 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 12:04:40.039274 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:04:40.046813 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.043239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:37021 (local address 127.0.203.129:36115)
0619 12:04:40.043320 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:40.043323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:40.043846 (+ 523us) server_negotiation.cc:407] Connection header received
0619 12:04:40.043909 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:40.043910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:40.043927 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:40.044004 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:40.044161 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.044948 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.045812 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.046155 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.046230 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.046550 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:40.046592 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:40.046641 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:40.046693 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 12:04:40.046725 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:04:40.046846 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.043123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:36115 (local address 127.0.203.131:37021)
0619 12:04:40.043297 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:40.043308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:40.043844 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:40.044024 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:40.044025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:40.044027 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:40.044090 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:40.044092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.045092 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.045093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:40.045726 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:40.045728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.046287 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.046289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:40.046408 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.046432 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:40.046648 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:40.046649 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:04:40.046690 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 12:04:40.046735 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260619 12:04:40.049734 2226 raft_consensus.cc:1261] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Refusing update from remote peer cd4c5a80173f4268952654008e3278d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:40.049862 2086 raft_consensus.cc:1261] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Refusing update from remote peer cd4c5a80173f4268952654008e3278d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:40.050256 2441 consensus_queue.cc:1035] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5e5b70542241aca388fac9403983e9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 38605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:40.050619 2467 consensus_queue.cc:1035] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6d719b125444d189d4880f4119d295d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 36115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:40.064230 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.058922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:33591 (local address 127.0.203.129:60221)
0619 12:04:40.059201 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:40.059217 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:40.059302 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:40.059569 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:40.059570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:40.059573 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:40.059635 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:40.059637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.060794 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.060801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:40.061864 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:40.061869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.063083 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.063085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:40.063179 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.063423 (+ 244us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:40.063979 (+ 556us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:40.063983 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:04:40.064035 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:04:40.064121 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":136,"threads_started":1}
I20260619 12:04:40.064203 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.059043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:60221 (local address 127.0.203.131:33591)
0619 12:04:40.059098 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:40.059101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:40.059271 (+ 170us) server_negotiation.cc:407] Connection header received
0619 12:04:40.059446 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:40.059447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:40.059460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:40.059497 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:40.059723 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.060549 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.061976 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.062319 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.062440 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.063694 (+ 1254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:40.063756 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:40.063797 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:40.064046 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 12:04:40.064078 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:04:40.083217 1929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:40.083432 1929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:40.084317 1929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e5949bd0b794cdeb3790f048e5d6223: Sending DeleteTablet for 3 replicas of tablet 0ded7dd6c7a446fdb510efaf92dc92d9
I20260619 12:04:40.084856 2061 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ded7dd6c7a446fdb510efaf92dc92d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:40 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42004
I20260619 12:04:40.084856 2346 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ded7dd6c7a446fdb510efaf92dc92d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:40 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:44700
I20260619 12:04:40.084985 2459 tablet_replica.cc:330] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: stopping tablet replica
I20260619 12:04:40.084990 2461 tablet_replica.cc:330] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: stopping tablet replica
I20260619 12:04:40.085048 2459 raft_consensus.cc:2229] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:04:40.085075 2461 raft_consensus.cc:2229] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:40.085127 2461 raft_consensus.cc:2258] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:40.085165 2459 raft_consensus.cc:2258] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:40.085677 2459 ts_tablet_manager.cc:1905] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:40.085706 2461 ts_tablet_manager.cc:1905] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:40.088285 2459 ts_tablet_manager.cc:1918] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:40.088368 2459 log.cc:1196] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/0ded7dd6c7a446fdb510efaf92dc92d9
I20260619 12:04:40.088390 2461 ts_tablet_manager.cc:1918] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:40.088475 2461 log.cc:1196] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/0ded7dd6c7a446fdb510efaf92dc92d9
I20260619 12:04:40.088630 2459 ts_tablet_manager.cc:1939] T 0ded7dd6c7a446fdb510efaf92dc92d9 P cd4c5a80173f4268952654008e3278d8: Deleting consensus metadata
I20260619 12:04:40.088873 2461 ts_tablet_manager.cc:1939] T 0ded7dd6c7a446fdb510efaf92dc92d9 P c6d719b125444d189d4880f4119d295d: Deleting consensus metadata
I20260619 12:04:40.089432 1916 catalog_manager.cc:4841] TS c6d719b125444d189d4880f4119d295d (127.0.203.129:36115): tablet 0ded7dd6c7a446fdb510efaf92dc92d9 (table test-table [id=e3d2c9814d97441f8ad0a35c5f55f6cc]) successfully deleted
I20260619 12:04:40.089588 1914 catalog_manager.cc:4841] TS cd4c5a80173f4268952654008e3278d8 (127.0.203.131:33591): tablet 0ded7dd6c7a446fdb510efaf92dc92d9 (table test-table [id=e3d2c9814d97441f8ad0a35c5f55f6cc]) successfully deleted
I20260619 12:04:40.089839 2206 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ded7dd6c7a446fdb510efaf92dc92d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:40 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:44884
I20260619 12:04:40.090852 2460 tablet_replica.cc:330] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: stopping tablet replica
I20260619 12:04:40.090965 2460 raft_consensus.cc:2229] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:40.091032 2460 raft_consensus.cc:2258] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:40.091517 2460 ts_tablet_manager.cc:1905] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:40.093758 2460 ts_tablet_manager.cc:1918] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:40.093824 2460 log.cc:1196] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/0ded7dd6c7a446fdb510efaf92dc92d9
I20260619 12:04:40.094027 2460 ts_tablet_manager.cc:1939] T 0ded7dd6c7a446fdb510efaf92dc92d9 P 4e5e5b70542241aca388fac9403983e9: Deleting consensus metadata
I20260619 12:04:40.094583 1915 catalog_manager.cc:4841] TS 4e5e5b70542241aca388fac9403983e9 (127.0.203.130:38605): tablet 0ded7dd6c7a446fdb510efaf92dc92d9 (table test-table [id=e3d2c9814d97441f8ad0a35c5f55f6cc]) successfully deleted
Jun 19 12:04:40 dist-test-slave-44vj krb5kdc[1884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:40.105706 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.091564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42020 (local address 127.0.203.129:36115)
0619 12:04:40.091626 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:40.091629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:40.092097 (+ 468us) server_negotiation.cc:407] Connection header received
0619 12:04:40.092241 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:40.092242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:40.092257 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:40.092290 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:40.093157 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.093878 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.094974 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.095293 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.095340 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.100910 (+ 5570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:40.100931 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:40.100932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:40.100946 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:40.103515 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:40.104322 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:40.104328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:40.104328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:40.104431 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:40.104729 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:40.104730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:40.104730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:40.104943 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:40.105124 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:40.105268 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 12:04:40.105547 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 12:04:40.106369 2126 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42020
I20260619 12:04:40.117815 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:40.107932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.0.203.190:40467)
0619 12:04:40.107989 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:40.107991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:40.108407 (+ 416us) server_negotiation.cc:407] Connection header received
0619 12:04:40.108543 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:40.108544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:40.108559 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:40.108591 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:40.109811 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.110856 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.112063 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:40.112496 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:40.112558 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:40.114524 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:40.114526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:40.114526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:40.114536 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:40.116456 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:40.116892 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:40.116893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:40.116893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:40.116949 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:40.117138 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:40.117139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:40.117139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:40.117284 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:40.117366 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:40.117593 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 12:04:40.117708 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260619 12:04:40.118783 1929 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41106
I20260619 12:04:40.120008 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 2001
I20260619 12:04:40.126669 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 2141
I20260619 12:04:40.131932 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 2281
I20260619 12:04:40.137692 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 1897
2026-06-19T12:04:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2591 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:40 dist-test-slave-44vj krb5kdc[2502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:40 dist-test-slave-44vj krb5kdc[2502](info): set up 2 sockets
Jun 19 12:04:40 dist-test-slave-44vj krb5kdc[2502](info): commencing operation
krb5kdc: starting...
W20260619 12:04:41.103323 814 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:41.230048 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34681
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:34681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:41.312574 2515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:41.312821 2515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:41.312845 2515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:41.314123 2515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:41.314183 2515 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:04:41.314195 2515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:41.314208 2515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:41.314218 2515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:41.314230 2515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:41.315913 2515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:34681
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:41.316155 2515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:41.316359 2515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:41.321559 2520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:41.321683 2515 server_base.cc:1034] running on GCE node
W20260619 12:04:41.321573 2521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:41.321573 2523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:41.322147 2515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:41.322499 2515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:41.323648 2515 hybrid_clock.cc:648] HybridClock initialized: now 1781870681323626 us; error 52 us; skew 500 ppm
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:41.326411 2515 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:41.326932 2515 webserver.cc:460] Webserver started at http://127.0.203.190:36421/ using document root <none> and password file <none>
I20260619 12:04:41.327143 2515 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:41.327194 2515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:41.327323 2515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:41.328449 2515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "10795a945a964778aeccd477229a7bb4"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "686e751993813bc46c0c077a9534a877"
server_key_iv: "623829050b16b86939bb25a75456b674"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.328784 2515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "10795a945a964778aeccd477229a7bb4"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "686e751993813bc46c0c077a9534a877"
server_key_iv: "623829050b16b86939bb25a75456b674"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.330205 2515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:41.331143 2530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.331490 2515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:41.331607 2515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "10795a945a964778aeccd477229a7bb4"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "686e751993813bc46c0c077a9534a877"
server_key_iv: "623829050b16b86939bb25a75456b674"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.331686 2515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:41.350795 2515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:41.353082 2515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:41.353219 2515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:41.358436 2515 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:34681
I20260619 12:04:41.358582 2592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:34681 every 8 connection(s)
I20260619 12:04:41.358880 2515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:41.359786 2593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:41.362497 2593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Bootstrap starting.
I20260619 12:04:41.363233 2593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:41.363620 2593 log.cc:824] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:41.364672 2593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: No bootstrap required, opened a new log
I20260619 12:04:41.366052 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 2515
I20260619 12:04:41.366153 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:41.366335 814 external_mini_cluster.cc:1428] Setting key 42445f33b9ab11ee46262d50bf1e825d
I20260619 12:04:41.367236 2593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10795a945a964778aeccd477229a7bb4" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34681 } }
I20260619 12:04:41.367508 2593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:41.367558 2593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10795a945a964778aeccd477229a7bb4, State: Initialized, Role: FOLLOWER
I20260619 12:04:41.367748 2593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10795a945a964778aeccd477229a7bb4" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34681 } }
I20260619 12:04:41.367841 2593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:41.367879 2593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:41.367918 2593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:41.368717 2593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10795a945a964778aeccd477229a7bb4" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34681 } }
I20260619 12:04:41.368835 2593 leader_election.cc:302] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10795a945a964778aeccd477229a7bb4; no voters:
I20260619 12:04:41.369032 2593 leader_election.cc:288] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:41.369235 2599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:41.369313 2593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:41.369452 2599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [term 1 LEADER]: Becoming Leader. State: Replica: 10795a945a964778aeccd477229a7bb4, State: Running, Role: LEADER
I20260619 12:04:41.369557 2599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10795a945a964778aeccd477229a7bb4" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34681 } }
I20260619 12:04:41.370023 2598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10795a945a964778aeccd477229a7bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10795a945a964778aeccd477229a7bb4" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34681 } } }
I20260619 12:04:41.370183 2598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:41.370483 2599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10795a945a964778aeccd477229a7bb4. Latest consensus state: current_term: 1 leader_uuid: "10795a945a964778aeccd477229a7bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10795a945a964778aeccd477229a7bb4" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34681 } } }
I20260619 12:04:41.370609 2599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:41.372051 2607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:41.372676 2607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:41.375545 2607 catalog_manager.cc:1347] Generated new cluster ID: 35cd9729602d4908a292c5e6ea42b289
I20260619 12:04:41.375643 2607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:41.379529 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.367243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.0.203.190:34681)
0619 12:04:41.367490 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.367501 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.367540 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:04:41.367714 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.367723 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.367789 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.368925 (+ 1136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.369057 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.370474 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.371621 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.372223 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.372369 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.375722 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.375728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.375728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.375743 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.377658 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.378111 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.378117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.378117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.378280 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.378476 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.378477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.378477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.378779 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.378949 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.379061 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.379231 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":148,"threads_started":1}
I20260619 12:04:41.387822 2607 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:41.388643 2607 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:41.400118 2607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Generated new TSK 0
I20260619 12:04:41.400362 2607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:41.433586 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34681
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:41.520313 2619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:41.520551 2619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:41.520588 2619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:41.521898 2619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:41.521960 2619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:41.521992 2619 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:04:41.522008 2619 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:04:41.522042 2619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:41.523782 2619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:34681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:41.524075 2619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:41.524302 2619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:41.529742 2627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:41.529776 2625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:41.529793 2624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:41.530242 2619 server_base.cc:1034] running on GCE node
I20260619 12:04:41.530498 2619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:41.530839 2619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:41.532044 2619 hybrid_clock.cc:648] HybridClock initialized: now 1781870681532005 us; error 53 us; skew 500 ppm
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:41.535109 2619 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:41.535851 2619 webserver.cc:460] Webserver started at http://127.0.203.129:39567/ using document root <none> and password file <none>
I20260619 12:04:41.536104 2619 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:41.536155 2619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:41.536281 2619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:41.537370 2619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "72a9500749b64e5fb78892d84479f474"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "9434bd573f4ed15da67cd52320f971ae"
server_key_iv: "9f007917bab61adeec18e8b1c77707df"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.537720 2619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "72a9500749b64e5fb78892d84479f474"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "9434bd573f4ed15da67cd52320f971ae"
server_key_iv: "9f007917bab61adeec18e8b1c77707df"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.539117 2619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:41.540203 2634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.540520 2619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:41.540630 2619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "72a9500749b64e5fb78892d84479f474"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "9434bd573f4ed15da67cd52320f971ae"
server_key_iv: "9f007917bab61adeec18e8b1c77707df"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.540706 2619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:41.571450 2619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:41.573737 2619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:41.573882 2619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:41.574513 2619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:41.574568 2619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.574596 2619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:41.574613 2619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:41.581959 2619 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:33427
I20260619 12:04:41.582058 2749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:33427 every 8 connection(s)
I20260619 12:04:41.582643 2619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:04:41.585776 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.575097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:52677 (local address 127.0.203.190:34681)
0619 12:04:41.575176 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.575178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.575501 (+ 323us) server_negotiation.cc:407] Connection header received
0619 12:04:41.576217 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.576219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.576238 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.576322 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.577155 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.577711 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.578855 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.579074 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.579135 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.581881 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.581884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.581884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.581898 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.583864 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.584711 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.584714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.584714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.584791 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.585083 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.585084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.585084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.585221 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.585282 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.585579 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.585682 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:04:41.585959 2643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.575056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34681 (local address 127.0.203.129:52677)
0619 12:04:41.575286 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.575339 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.576063 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.576440 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.576445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.576794 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:41.577051 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.577057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.577812 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.577813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.578764 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.578767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.579162 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.579163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.579289 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.579918 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:41.579928 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:41.581720 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:41.584177 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.584185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.584188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.584548 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.584890 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.584891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.584892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.585014 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.585278 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:41.585286 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:41.585443 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.585545 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.585734 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":127,"threads_started":1}
I20260619 12:04:41.586539 2750 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34681
I20260619 12:04:41.586664 2750 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:41.586884 2750 heartbeater.cc:511] Master 127.0.203.190:34681 requested a full tablet report, sending...
I20260619 12:04:41.587556 2546 ts_manager.cc:194] Registered new tserver with Master: 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427)
I20260619 12:04:41.588289 2546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:52677
I20260619 12:04:41.590474 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 2619
I20260619 12:04:41.590571 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:41.590732 814 external_mini_cluster.cc:1428] Setting key be1e977d1564fb778c56ff090ad35b84
I20260619 12:04:41.595325 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.587788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34681 (local address 127.0.203.129:51047)
0619 12:04:41.588542 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.588553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.588624 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.589222 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.589223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.589393 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:41.589444 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.589446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.590140 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.590141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.590704 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.590706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.591432 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.591434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.591513 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.592082 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:41.592088 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:41.592527 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:41.594289 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.594290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.594290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.594477 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.594690 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.594690 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.594691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.594747 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.595003 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:41.595003 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:41.595079 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.595135 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.595240 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":138,"threads_started":1}
I20260619 12:04:41.595350 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.587855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:51047 (local address 127.0.203.190:34681)
0619 12:04:41.587907 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.587909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.588624 (+ 715us) server_negotiation.cc:407] Connection header received
0619 12:04:41.588681 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.588682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.588693 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.588748 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.589541 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.590045 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.590809 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.591107 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.591197 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.592626 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.592628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.592628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.592636 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.594174 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.594557 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.594558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.594559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.594621 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.594804 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.594805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.594805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.594941 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.595008 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.595177 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.595262 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:41.618665 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34681
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:41.705343 2759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:41.705569 2759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:41.705596 2759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:41.706884 2759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:41.706935 2759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:41.706959 2759 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:04:41.706969 2759 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:04:41.706985 2759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:41.708820 2759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:34681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:41.709148 2759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:41.709396 2759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:41.714681 2765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:41.714720 2764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:41.714681 2767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:41.714986 2759 server_base.cc:1034] running on GCE node
I20260619 12:04:41.715214 2759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:41.715541 2759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:41.716735 2759 hybrid_clock.cc:648] HybridClock initialized: now 1781870681716705 us; error 46 us; skew 500 ppm
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:41.719491 2759 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:41.720073 2759 webserver.cc:460] Webserver started at http://127.0.203.130:45055/ using document root <none> and password file <none>
I20260619 12:04:41.720299 2759 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:41.720351 2759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:41.720485 2759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:41.721566 2759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "9cd8adc6b7bd45508bc153ce2c3b9504"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "557aaab7c42c6c2dbf38beb22b471d1a"
server_key_iv: "a7498b3e4bc39d8c62eb7fbdf7cdf134"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.721908 2759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "9cd8adc6b7bd45508bc153ce2c3b9504"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "557aaab7c42c6c2dbf38beb22b471d1a"
server_key_iv: "a7498b3e4bc39d8c62eb7fbdf7cdf134"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.723409 2759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:41.724282 2774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.724458 2759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:41.724558 2759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "9cd8adc6b7bd45508bc153ce2c3b9504"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "557aaab7c42c6c2dbf38beb22b471d1a"
server_key_iv: "a7498b3e4bc39d8c62eb7fbdf7cdf134"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.724630 2759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:41.736339 2759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:41.738402 2759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:41.738540 2759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:41.739091 2759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:41.739135 2759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.739172 2759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:41.739203 2759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.746658 2759 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:35627
I20260619 12:04:41.746771 2889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:35627 every 8 connection(s)
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:41.747090 2759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:41.750325 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.739953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:38301 (local address 127.0.203.190:34681)
0619 12:04:41.740011 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.740013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.740263 (+ 250us) server_negotiation.cc:407] Connection header received
0619 12:04:41.740927 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.740928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.740943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.740998 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.742035 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.742795 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.744597 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.744869 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.744939 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.747697 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.747700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.747700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.747712 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.749069 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.749502 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.749503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.749504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.749554 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.749765 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.749765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.749766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.749876 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.749927 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.750182 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.750252 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:04:41.750563 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.739907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34681 (local address 127.0.203.130:38301)
0619 12:04:41.740113 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.740159 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.740824 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.741087 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.741093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.741559 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:41.741893 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.741902 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.742927 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.742929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.744504 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.744509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.744948 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.744949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.745110 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.745849 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:41.745860 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:41.747573 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:41.749163 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.749170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.749172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.749423 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.749606 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.749607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.749607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.749711 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.749924 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:41.749928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:41.750065 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.750158 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.750318 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260619 12:04:41.751161 2890 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34681
I20260619 12:04:41.751256 2890 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:41.751503 2890 heartbeater.cc:511] Master 127.0.203.190:34681 requested a full tablet report, sending...
I20260619 12:04:41.751965 2547 ts_manager.cc:194] Registered new tserver with Master: 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627)
I20260619 12:04:41.752377 2547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:38301
I20260619 12:04:41.754783 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 2759
I20260619 12:04:41.754886 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:41.755057 814 external_mini_cluster.cc:1428] Setting key 7f50809dee06460795129498016d3730
I20260619 12:04:41.758440 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.752371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34681 (local address 127.0.203.130:47717)
0619 12:04:41.752546 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.752557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.752621 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.752766 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.752767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.752964 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:41.753025 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.753026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.753685 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.753686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.754217 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.754218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.754538 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.754538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.754623 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.755086 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:41.755092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:41.755472 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:41.757081 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.757082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.757082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.757326 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.757629 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.757630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.757631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.757693 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.758111 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:41.758112 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:41.758169 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.758245 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.758352 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260619 12:04:41.758502 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.752472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:47717 (local address 127.0.203.190:34681)
0619 12:04:41.752522 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.752523 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.752616 (+ 93us) server_negotiation.cc:407] Connection header received
0619 12:04:41.752686 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.752687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.752697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.752760 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.753087 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.753598 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.754280 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.754484 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.754536 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.755614 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.755616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.755616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.755625 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.756987 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.757446 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.757447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.757448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.757532 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.757760 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.757760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.757761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.757984 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.758076 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.758299 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.758406 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:41.782320 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34681
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:41.870986 2899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:41.871268 2899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:41.871308 2899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:41.872653 2899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:41.872704 2899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:41.872730 2899 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:04:41.872756 2899 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:04:41.872782 2899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:41.874357 2899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:34681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:41.874608 2899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:41.874867 2899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:41.880478 2904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:41.880504 2907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:41.880647 2899 server_base.cc:1034] running on GCE node
W20260619 12:04:41.880478 2905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:41.880985 2899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:41.881283 2899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:41.882484 2899 hybrid_clock.cc:648] HybridClock initialized: now 1781870681882467 us; error 35 us; skew 500 ppm
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:41.885272 2899 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:41.885849 2899 webserver.cc:460] Webserver started at http://127.0.203.131:36761/ using document root <none> and password file <none>
I20260619 12:04:41.886094 2899 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:41.886159 2899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:41.886298 2899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:41.887506 2899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "0aba3dbb406e45309472fa7fef6e2a72"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "27d631a03717b43c45d748fc7a7c9397"
server_key_iv: "5695a8af499d624c7e67f7c9adcffb72"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.887861 2899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "0aba3dbb406e45309472fa7fef6e2a72"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "27d631a03717b43c45d748fc7a7c9397"
server_key_iv: "5695a8af499d624c7e67f7c9adcffb72"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.889232 2899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:41.889992 2914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.890185 2899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:41.890300 2899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "0aba3dbb406e45309472fa7fef6e2a72"
format_stamp: "Formatted at 2026-06-19 12:04:41 on dist-test-slave-44vj"
server_key: "27d631a03717b43c45d748fc7a7c9397"
server_key_iv: "5695a8af499d624c7e67f7c9adcffb72"
server_key_version: "encryptionkey@0"
I20260619 12:04:41.890383 2899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:41.933473 2899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:41.935742 2899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:41.935868 2899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:41.936480 2899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:41.936527 2899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:41.936553 2899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:41.936565 2899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:41.944466 2899 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:36123
I20260619 12:04:41.944571 3029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:36123 every 8 connection(s)
I20260619 12:04:41.945065 2899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:04:41.948067 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.937073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:44957 (local address 127.0.203.190:34681)
0619 12:04:41.937150 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.937153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.937406 (+ 253us) server_negotiation.cc:407] Connection header received
0619 12:04:41.937949 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.937951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.937992 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.938082 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.938982 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.939793 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.941331 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.941631 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.941710 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.944847 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.944849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.944849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.944860 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.946499 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.947026 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.947027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.947028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.947090 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.947399 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.947400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.947401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.947552 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.947622 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.947913 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.947992 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:04:41.948271 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.937050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34681 (local address 127.0.203.131:44957)
0619 12:04:41.937256 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.937300 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.937818 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.938186 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.938190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.938563 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:41.938868 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.938874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.939904 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.939904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.941203 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.941207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.941742 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.941743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.941895 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.942636 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:41.942645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:41.944583 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:41.946621 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.946628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.946630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.946926 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.947151 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.947152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.947152 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.947259 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.947619 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:41.947624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:41.947769 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.947882 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.948036 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":133,"threads_started":1}
I20260619 12:04:41.948453 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 2899
I20260619 12:04:41.948534 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:41.948740 814 external_mini_cluster.cc:1428] Setting key 0dfc1b8a1d3d9e166ffd62d65056b9bd
I20260619 12:04:41.948843 3030 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34681
I20260619 12:04:41.948947 3030 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:41.949162 3030 heartbeater.cc:511] Master 127.0.203.190:34681 requested a full tablet report, sending...
I20260619 12:04:41.950059 2547 ts_manager.cc:194] Registered new tserver with Master: 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123)
I20260619 12:04:41.950565 2547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:44957
I20260619 12:04:41.956274 3036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.950072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34681 (local address 127.0.203.131:41409)
0619 12:04:41.950290 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.950302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.950355 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.950871 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.950873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.951084 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:41.951153 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.951155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.951914 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.951915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.952482 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.952484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.952830 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.952831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.952913 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.953429 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:41.953434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:41.953772 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:41.955219 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.955220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.955221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.955455 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.955644 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:41.955645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:41.955645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:41.955708 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:41.955925 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:41.955926 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:41.955980 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.956047 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.956169 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":128,"threads_started":1}
I20260619 12:04:41.956271 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.950165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:41409 (local address 127.0.203.190:34681)
0619 12:04:41.950744 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.950746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.950756 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:04:41.950777 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.950778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.950788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.950849 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.951227 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.951819 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.952557 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.952769 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.952825 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.953859 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.953861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.953861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.953869 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.955135 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.955536 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.955538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.955538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.955592 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.955757 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.955758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.955759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.955873 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.955930 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.956074 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.956156 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555}
I20260619 12:04:41.960717 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:04:41 dist-test-slave-44vj krb5kdc[2502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:41.986836 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.978947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.0.203.190:34681)
0619 12:04:41.979037 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.979039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.979048 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:04:41.979079 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.979080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.979091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.979140 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:41.979848 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.980379 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.981133 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.981323 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.981380 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.984190 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:41.984196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:41.984196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:41.984208 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:41.985640 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.986048 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.986049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.986050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.986131 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:41.986334 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:41.986334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:41.986335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:41.986471 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:41.986553 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.986682 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.986763 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:04:41.988790 2547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:41.990808 2547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:41.998279 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.993857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35627 (local address 127.0.0.1:40702)
0619 12:04:41.994083 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.994127 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.994217 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.994484 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.994489 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.994492 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:41.994611 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.994614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.995829 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.995830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.996977 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.996980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.997912 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.997913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.998004 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.998021 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.998093 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.998163 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":81,"threads_started":1}
I20260619 12:04:41.998284 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.993971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.0.203.130:35627)
0619 12:04:41.994264 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.994269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.994281 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:04:41.994323 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.994325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.994341 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.994454 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:41.994829 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.995747 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.997085 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.997834 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.997903 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.997975 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.998113 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.998163 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
I20260619 12:04:41.998833 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.993901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:36123 (local address 127.0.0.1:42382)
0619 12:04:41.994625 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.994637 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.994705 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.994898 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.994899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.994901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:41.994955 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.994957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.996386 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.996387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.997446 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.997449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.998581 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.998582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.998656 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.998661 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.998712 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.998766 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":94,"threads_started":1}
I20260619 12:04:41.998988 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.994154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.0.203.131:36123)
0619 12:04:41.994415 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.994418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.994726 (+ 308us) server_negotiation.cc:407] Connection header received
0619 12:04:41.994782 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.994785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.994801 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.994884 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:41.995036 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.996292 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.997545 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.998335 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.998576 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.998648 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.998740 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.998788 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":100,"threads_started":1}
I20260619 12:04:41.999054 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.993861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33427 (local address 127.0.0.1:32954)
0619 12:04:41.994385 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:41.994395 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:41.994453 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:41.994992 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:41.994994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:41.994997 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:41.995062 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.995064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.996379 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.996380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.997356 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:41.997358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.998686 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.998687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:41.998761 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.998768 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:04:41.998808 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 12:04:41.998871 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":110,"threads_started":1}
I20260619 12:04:41.999213 2824 tablet_service.cc:1467] Processing CreateTablet for tablet c1a00d3d6cf2453db4ea9f118bf21335 (DEFAULT_TABLE table=test-table [id=1188fd6e880f42a6bef06e59a2ca553d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:41.999482 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:41.994189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.0.203.129:33427)
0619 12:04:41.994390 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:41.994397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:41.994786 (+ 389us) server_negotiation.cc:407] Connection header received
0619 12:04:41.994836 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:41.994838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:41.994857 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:41.994942 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:41.995148 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.996270 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.997446 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:41.998328 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:41.998411 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:41.999267 (+ 856us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:41.999295 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 12:04:41.999341 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":94,"threads_started":1}
I20260619 12:04:41.999709 2824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1a00d3d6cf2453db4ea9f118bf21335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.000187 2684 tablet_service.cc:1467] Processing CreateTablet for tablet c1a00d3d6cf2453db4ea9f118bf21335 (DEFAULT_TABLE table=test-table [id=1188fd6e880f42a6bef06e59a2ca553d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:42.000187 2964 tablet_service.cc:1467] Processing CreateTablet for tablet c1a00d3d6cf2453db4ea9f118bf21335 (DEFAULT_TABLE table=test-table [id=1188fd6e880f42a6bef06e59a2ca553d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:42.000582 2964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1a00d3d6cf2453db4ea9f118bf21335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.000582 2684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1a00d3d6cf2453db4ea9f118bf21335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.003439 3053 tablet_bootstrap.cc:492] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Bootstrap starting.
I20260619 12:04:42.004287 3053 tablet_bootstrap.cc:654] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.004449 3054 tablet_bootstrap.cc:492] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Bootstrap starting.
I20260619 12:04:42.004817 3053 log.cc:824] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:42.005440 3054 tablet_bootstrap.cc:654] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.005602 3053 tablet_bootstrap.cc:492] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: No bootstrap required, opened a new log
I20260619 12:04:42.005676 3053 ts_tablet_manager.cc:1397] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:04:42.005790 3054 log.cc:824] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:42.006171 3055 tablet_bootstrap.cc:492] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Bootstrap starting.
I20260619 12:04:42.006626 3054 tablet_bootstrap.cc:492] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: No bootstrap required, opened a new log
I20260619 12:04:42.006701 3054 ts_tablet_manager.cc:1397] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:04:42.006862 3055 tablet_bootstrap.cc:654] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.007155 3055 log.cc:824] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:42.007757 3053 raft_consensus.cc:348] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.007941 3053 raft_consensus.cc:374] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.007974 3053 raft_consensus.cc:729] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd8adc6b7bd45508bc153ce2c3b9504, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.008169 3055 tablet_bootstrap.cc:492] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: No bootstrap required, opened a new log
I20260619 12:04:42.008132 3053 consensus_queue.cc:260] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.008252 3055 ts_tablet_manager.cc:1397] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:04:42.008471 3053 ts_tablet_manager.cc:1428] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 12:04:42.008607 2890 heartbeater.cc:503] Master 127.0.203.190:34681 was elected leader, sending a full tablet report...
I20260619 12:04:42.008706 3054 raft_consensus.cc:348] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.008910 3054 raft_consensus.cc:374] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.008944 3054 raft_consensus.cc:729] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba3dbb406e45309472fa7fef6e2a72, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.009095 3054 consensus_queue.cc:260] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.009411 3054 ts_tablet_manager.cc:1428] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:42.009519 3030 heartbeater.cc:503] Master 127.0.203.190:34681 was elected leader, sending a full tablet report...
I20260619 12:04:42.010238 3055 raft_consensus.cc:348] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.010412 3055 raft_consensus.cc:374] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.010465 3055 raft_consensus.cc:729] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a9500749b64e5fb78892d84479f474, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.010584 3055 consensus_queue.cc:260] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.010845 3055 ts_tablet_manager.cc:1428] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:04:42.010933 2750 heartbeater.cc:503] Master 127.0.203.190:34681 was elected leader, sending a full tablet report...
W20260619 12:04:42.083812 2751 tablet.cc:2369] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:42.109795 3061 raft_consensus.cc:482] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:42.109958 3061 raft_consensus.cc:504] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.110373 3061 leader_election.cc:288] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627), 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123)
I20260619 12:04:42.113842 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.110825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:42281 (local address 127.0.203.130:35627)
0619 12:04:42.110873 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.110875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.110887 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:04:42.110969 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.110970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.110978 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.111074 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:42.111341 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.111915 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.113104 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.113474 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.113537 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.113546 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.113716 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.113758 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:04:42.113878 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.110771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:54087 (local address 127.0.203.131:36123)
0619 12:04:42.110829 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.110831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.110897 (+ 66us) server_negotiation.cc:407] Connection header received
0619 12:04:42.110968 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.110969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.110978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.111074 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:42.111427 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.112013 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.113121 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.113505 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.113584 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.113595 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.113754 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.113803 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:04:42.113931 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.110682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35627 (local address 127.0.203.129:42281)
0619 12:04:42.110830 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.110838 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.110901 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.111126 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.111128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.111132 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:42.111244 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.111247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.112013 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.112015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.112993 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.112996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.113559 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.113560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.113640 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.113646 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.113693 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.113751 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":43,"thread_start_us":75,"threads_started":1}
I20260619 12:04:42.114120 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.110642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:36123 (local address 127.0.203.129:54087)
0619 12:04:42.110809 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.110823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.110901 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.111126 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.111128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.111132 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:42.111266 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.111267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.112117 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.112118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.113012 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.113015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.113601 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.113602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.113669 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.113674 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.113732 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.113792 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":32,"thread_start_us":84,"threads_started":1}
I20260619 12:04:42.114367 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a00d3d6cf2453db4ea9f118bf21335" candidate_uuid: "72a9500749b64e5fb78892d84479f474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" is_pre_election: true
I20260619 12:04:42.114571 2844 raft_consensus.cc:2454] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72a9500749b64e5fb78892d84479f474 in term 0.
I20260619 12:04:42.114866 2984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a00d3d6cf2453db4ea9f118bf21335" candidate_uuid: "72a9500749b64e5fb78892d84479f474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba3dbb406e45309472fa7fef6e2a72" is_pre_election: true
I20260619 12:04:42.114877 2635 leader_election.cc:302] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72a9500749b64e5fb78892d84479f474, 9cd8adc6b7bd45508bc153ce2c3b9504; no voters:
I20260619 12:04:42.115010 2984 raft_consensus.cc:2454] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72a9500749b64e5fb78892d84479f474 in term 0.
I20260619 12:04:42.115111 3061 raft_consensus.cc:2781] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:42.115165 3061 raft_consensus.cc:482] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:42.115188 3061 raft_consensus.cc:3037] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.115998 3061 raft_consensus.cc:504] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.116156 3061 leader_election.cc:288] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [CANDIDATE]: Term 1 election: Requested vote from peers 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627), 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123)
I20260619 12:04:42.116366 2984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a00d3d6cf2453db4ea9f118bf21335" candidate_uuid: "72a9500749b64e5fb78892d84479f474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba3dbb406e45309472fa7fef6e2a72"
I20260619 12:04:42.116389 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a00d3d6cf2453db4ea9f118bf21335" candidate_uuid: "72a9500749b64e5fb78892d84479f474" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504"
I20260619 12:04:42.116472 2984 raft_consensus.cc:3037] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.116464 2844 raft_consensus.cc:3037] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.117290 2984 raft_consensus.cc:2454] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72a9500749b64e5fb78892d84479f474 in term 1.
I20260619 12:04:42.117489 2638 leader_election.cc:302] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba3dbb406e45309472fa7fef6e2a72, 72a9500749b64e5fb78892d84479f474; no voters:
I20260619 12:04:42.117573 3061 raft_consensus.cc:2781] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:42.117573 2844 raft_consensus.cc:2454] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72a9500749b64e5fb78892d84479f474 in term 1.
I20260619 12:04:42.117705 3061 raft_consensus.cc:686] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 1 LEADER]: Becoming Leader. State: Replica: 72a9500749b64e5fb78892d84479f474, State: Running, Role: LEADER
I20260619 12:04:42.117820 3061 consensus_queue.cc:237] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.118816 2547 catalog_manager.cc:5495] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72a9500749b64e5fb78892d84479f474 (127.0.203.129). New cstate: current_term: 1 leader_uuid: "72a9500749b64e5fb78892d84479f474" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } health_report { overall_health: UNKNOWN } } }
Jun 19 12:04:42 dist-test-slave-44vj krb5kdc[2502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:42.187665 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.180455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.0.203.190:34681)
0619 12:04:42.180533 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.180536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.180673 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:04:42.180752 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.180752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.180763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.180812 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:42.181338 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.181862 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.182534 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.182711 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.182766 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.185111 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:42.185112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:42.185113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:42.185121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:42.186473 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:42.186886 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:42.186887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:42.186887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:42.186954 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:42.187112 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:42.187112 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:42.187113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:42.187228 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:42.187285 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.187453 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.187539 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:04:42.187723 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.180369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34681 (local address 127.0.0.1:35272)
0619 12:04:42.180561 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.180575 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.180685 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.180821 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.180823 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.181196 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:42.181254 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.181256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.181929 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.181930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.182461 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.182462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.182768 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.182769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.182812 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.183418 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:42.183422 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:42.185018 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:42.186550 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:42.186558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:42.186561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:42.186824 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:42.187005 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:42.187006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:42.187006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:42.187065 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:42.187279 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:42.187286 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:42.187333 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.187431 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.187604 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":63,"thread_start_us":110,"threads_started":1}
I20260619 12:04:42.190285 2547 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:35272:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:04:42.190562 2547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:42.192377 2964 tablet_service.cc:1467] Processing CreateTablet for tablet 969b0ad9f85c40f3aedd9ff71b4f91af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae779e5e20ff48d7a4435f81b0b1483e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:04:42.192379 2684 tablet_service.cc:1467] Processing CreateTablet for tablet 969b0ad9f85c40f3aedd9ff71b4f91af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae779e5e20ff48d7a4435f81b0b1483e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:04:42.192579 2964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969b0ad9f85c40f3aedd9ff71b4f91af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.192605 2684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969b0ad9f85c40f3aedd9ff71b4f91af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.193931 3054 tablet_bootstrap.cc:492] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: Bootstrap starting.
I20260619 12:04:42.194139 3055 tablet_bootstrap.cc:492] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: Bootstrap starting.
I20260619 12:04:42.194290 2824 tablet_service.cc:1467] Processing CreateTablet for tablet 969b0ad9f85c40f3aedd9ff71b4f91af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae779e5e20ff48d7a4435f81b0b1483e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:04:42.194500 2824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969b0ad9f85c40f3aedd9ff71b4f91af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.194497 3054 tablet_bootstrap.cc:654] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.194653 3055 tablet_bootstrap.cc:654] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.195168 3054 tablet_bootstrap.cc:492] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: No bootstrap required, opened a new log
I20260619 12:04:42.195215 3054 ts_tablet_manager.cc:1397] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:42.195399 3054 raft_consensus.cc:348] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.195487 3054 raft_consensus.cc:374] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.195508 3054 raft_consensus.cc:729] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba3dbb406e45309472fa7fef6e2a72, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.195576 3054 consensus_queue.cc:260] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.195708 3054 ts_tablet_manager.cc:1428] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260619 12:04:42.195847 3031 tablet.cc:2369] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:42.195815 3060 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.195947 3060 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:42.196362 3053 tablet_bootstrap.cc:492] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: Bootstrap starting.
I20260619 12:04:42.197088 3053 tablet_bootstrap.cc:654] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.197603 3055 tablet_bootstrap.cc:492] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: No bootstrap required, opened a new log
I20260619 12:04:42.197659 3055 ts_tablet_manager.cc:1397] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 12:04:42.197842 3055 raft_consensus.cc:348] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.197918 3055 raft_consensus.cc:374] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.197921 3053 tablet_bootstrap.cc:492] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: No bootstrap required, opened a new log
I20260619 12:04:42.197937 3055 raft_consensus.cc:729] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a9500749b64e5fb78892d84479f474, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.197974 3053 ts_tablet_manager.cc:1397] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 12:04:42.197985 3055 consensus_queue.cc:260] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.198096 3055 ts_tablet_manager.cc:1428] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 12:04:42.198145 3061 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.198226 3061 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:42.198176 3053 raft_consensus.cc:348] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.198285 3053 raft_consensus.cc:374] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.198314 3053 raft_consensus.cc:729] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd8adc6b7bd45508bc153ce2c3b9504, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.198369 3053 consensus_queue.cc:260] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.198465 3053 ts_tablet_manager.cc:1428] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:42.198547 3059 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.198678 3059 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 12:04:42.248064 2891 tablet.cc:2369] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:42.257017 3059 raft_consensus.cc:482] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:42.257169 3059 raft_consensus.cc:504] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.257578 3059 leader_election.cc:288] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123)
I20260619 12:04:42.261883 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.257950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:33603 (local address 127.0.203.131:36123)
0619 12:04:42.257998 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.258000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.258127 (+ 127us) server_negotiation.cc:407] Connection header received
0619 12:04:42.258173 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.258174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.258185 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.258237 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:42.258449 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.259211 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.260843 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.261435 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.261527 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.261539 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.261750 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.261795 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:04:42.261883 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.257867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:44489 (local address 127.0.203.129:33427)
0619 12:04:42.257941 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.257944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.258041 (+ 97us) server_negotiation.cc:407] Connection header received
0619 12:04:42.258114 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.258116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.258126 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.258179 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:42.258389 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.259202 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.260844 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.261435 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.261527 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.261539 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.261749 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.261794 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:04:42.261940 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.257835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:36123 (local address 127.0.203.130:33603)
0619 12:04:42.258015 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.258047 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.258121 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.258243 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.258244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.258246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:42.258349 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.258352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.259324 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.259325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.260706 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.260710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.261550 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.261551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.261658 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.261668 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.261734 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.261791 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":103,"thread_start_us":118,"threads_started":1}
I20260619 12:04:42.261941 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.257783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33427 (local address 127.0.203.130:44489)
0619 12:04:42.257951 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.257963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.258055 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.258188 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.258189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.258192 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:42.258273 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.258276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.259323 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.259325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.260706 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.260710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.261549 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.261551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.261658 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.261666 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.261734 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.261791 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":60,"thread_start_us":75,"threads_started":1}
I20260619 12:04:42.262527 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969b0ad9f85c40f3aedd9ff71b4f91af" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474" is_pre_election: true
I20260619 12:04:42.262526 2984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969b0ad9f85c40f3aedd9ff71b4f91af" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba3dbb406e45309472fa7fef6e2a72" is_pre_election: true
I20260619 12:04:42.262692 2704 raft_consensus.cc:2454] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 0.
I20260619 12:04:42.262760 2984 raft_consensus.cc:2454] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 0.
I20260619 12:04:42.262943 2777 leader_election.cc:302] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72a9500749b64e5fb78892d84479f474, 9cd8adc6b7bd45508bc153ce2c3b9504; no voters:
I20260619 12:04:42.263134 3059 raft_consensus.cc:2781] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:42.263182 3059 raft_consensus.cc:482] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:42.263202 3059 raft_consensus.cc:3037] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.264072 3059 raft_consensus.cc:504] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.264228 3059 leader_election.cc:288] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 election: Requested vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123)
I20260619 12:04:42.264405 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969b0ad9f85c40f3aedd9ff71b4f91af" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474"
I20260619 12:04:42.264485 2704 raft_consensus.cc:3037] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.264572 2984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969b0ad9f85c40f3aedd9ff71b4f91af" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba3dbb406e45309472fa7fef6e2a72"
I20260619 12:04:42.264672 2984 raft_consensus.cc:3037] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.265276 2704 raft_consensus.cc:2454] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 1.
I20260619 12:04:42.265390 2984 raft_consensus.cc:2454] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 1.
I20260619 12:04:42.265480 2777 leader_election.cc:302] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72a9500749b64e5fb78892d84479f474, 9cd8adc6b7bd45508bc153ce2c3b9504; no voters:
I20260619 12:04:42.265568 3059 raft_consensus.cc:2781] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:42.265722 3059 raft_consensus.cc:686] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 LEADER]: Becoming Leader. State: Replica: 9cd8adc6b7bd45508bc153ce2c3b9504, State: Running, Role: LEADER
I20260619 12:04:42.265846 3059 consensus_queue.cc:237] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.266445 3077 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: TxnStatusTablet state changed. Reason: New leader 9cd8adc6b7bd45508bc153ce2c3b9504. Latest consensus state: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.266584 3077 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:04:42.266811 3079 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:04:42.266778 2547 catalog_manager.cc:5495] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130). New cstate: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } health_report { overall_health: UNKNOWN } } }
I20260619 12:04:42.266942 3079 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:04:42.291736 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.289269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35627 (local address 127.0.0.1:40716)
0619 12:04:42.289346 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.289357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.289406 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.289577 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.289577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.289579 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:42.289634 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.289637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.290254 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.290256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.290836 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.290837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.291172 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.291173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.291238 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.291258 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.291551 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.291560 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.291611 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.291665 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 12:04:42.291730 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.289363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40716 (local address 127.0.203.130:35627)
0619 12:04:42.289441 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.289443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.289452 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:04:42.289492 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.289493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.289513 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.289568 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:42.289695 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.290182 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.290911 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.291114 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.291167 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.291341 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.291488 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.291564 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.291617 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.291641 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:04:42.293901 2984 raft_consensus.cc:1261] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Refusing update from remote peer 9cd8adc6b7bd45508bc153ce2c3b9504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:42.293877 2704 raft_consensus.cc:1261] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Refusing update from remote peer 9cd8adc6b7bd45508bc153ce2c3b9504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:42.294288 3059 consensus_queue.cc:1035] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:42.294564 3077 consensus_queue.cc:1035] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:42.296096 3060 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: TxnStatusTablet state changed. Reason: New leader 9cd8adc6b7bd45508bc153ce2c3b9504. Latest consensus state: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.296068 3061 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: TxnStatusTablet state changed. Reason: New leader 9cd8adc6b7bd45508bc153ce2c3b9504. Latest consensus state: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.296176 3061 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:42.296176 3060 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:42.297690 3059 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.297801 3059 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:04:42.297845 3059 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.297883 3059 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 9cd8adc6b7bd45508bc153ce2c3b9504: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:04:42.298027 3082 mvcc.cc:205] Tried to move back new op lower bound from 7298542314672164864 to 7298542314561617920. Current Snapshot: MvccSnapshot[applied={T|T < 7298542314672164864}]
I20260619 12:04:42.298117 3084 mvcc.cc:205] Tried to move back new op lower bound from 7298542314672164864 to 7298542314561617920. Current Snapshot: MvccSnapshot[applied={T|T < 7298542314672164864}]
I20260619 12:04:42.298091 3060 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.298161 3060 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 0aba3dbb406e45309472fa7fef6e2a72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:42.298128 3061 tablet_replica.cc:439] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } } }
I20260619 12:04:42.298259 3061 tablet_replica.cc:442] T 969b0ad9f85c40f3aedd9ff71b4f91af P 72a9500749b64e5fb78892d84479f474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:04:42.298842 3083 mvcc.cc:205] Tried to move back new op lower bound from 7298542314672164864 to 7298542314561617920. Current Snapshot: MvccSnapshot[applied={T|T < 7298542314672164864}]
I20260619 12:04:42.303181 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.300322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.0.203.129:33427)
0619 12:04:42.300374 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.300376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.300454 (+ 78us) server_negotiation.cc:407] Connection header received
0619 12:04:42.300533 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.300533 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.300548 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.300611 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:42.300794 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.301270 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.302440 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.302641 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.302695 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.302862 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.302967 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.303028 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.303098 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.303117 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:04:42.309180 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.306641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:59985 (local address 127.0.203.130:35627)
0619 12:04:42.306695 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.306697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.306834 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:04:42.306959 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.306960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.306971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.307015 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:42.307160 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.307722 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.308503 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.308688 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.308743 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.308947 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.308980 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.309020 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.309070 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.309092 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:04:42.309208 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.306531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35627 (local address 127.0.203.129:59985)
0619 12:04:42.306751 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.306765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.306886 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.307027 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.307028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.307029 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:42.307099 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.307101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.307805 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.307807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.308417 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.308419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.308760 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.308761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.308829 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.308848 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.309027 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.309029 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.309069 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.309114 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":113,"threads_started":1}
I20260619 12:04:42.312994 2984 raft_consensus.cc:1261] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Refusing update from remote peer 72a9500749b64e5fb78892d84479f474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:42.313226 2844 raft_consensus.cc:1261] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Refusing update from remote peer 72a9500749b64e5fb78892d84479f474: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:42.313306 3061 consensus_queue.cc:1035] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:42.313536 3064 consensus_queue.cc:1035] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:42.324188 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.321122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:37151 (local address 127.0.203.129:33427)
0619 12:04:42.321165 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.321167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.321412 (+ 245us) server_negotiation.cc:407] Connection header received
0619 12:04:42.321471 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.321472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.321481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.321533 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:42.321741 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.322237 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.323554 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.323746 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.323805 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.323996 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.324023 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.324059 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.324113 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.324130 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:04:42.324270 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.321060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33427 (local address 127.0.203.130:37151)
0619 12:04:42.321332 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.321344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.321416 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.321540 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.321541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.321542 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:42.321628 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.321630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.322317 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.322317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.323473 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.323475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.323819 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.323820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.323889 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.323909 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.324065 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.324067 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.324111 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.324157 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":86,"threads_started":1}
I20260619 12:04:42.334645 2546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:42.334789 2546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:42.335668 2546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Sending DeleteTablet for 3 replicas of tablet c1a00d3d6cf2453db4ea9f118bf21335
I20260619 12:04:42.336133 2684 tablet_service.cc:1514] Processing DeleteTablet for tablet c1a00d3d6cf2453db4ea9f118bf21335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:42 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:32954
I20260619 12:04:42.336223 2964 tablet_service.cc:1514] Processing DeleteTablet for tablet c1a00d3d6cf2453db4ea9f118bf21335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:42 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42382
I20260619 12:04:42.336284 2824 tablet_service.cc:1514] Processing DeleteTablet for tablet c1a00d3d6cf2453db4ea9f118bf21335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:42 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:40702
I20260619 12:04:42.336673 2546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:42.336845 2546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:42.336772 3102 tablet_replica.cc:330] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: stopping tablet replica
I20260619 12:04:42.337149 3102 raft_consensus.cc:2229] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:04:42.337353 3102 raft_consensus.cc:2258] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:42.337975 2684 tablet_service.cc:1467] Processing CreateTablet for tablet 5d39d505fc384164941d9413c9026e37 (DEFAULT_TABLE table=test-table [id=3e707f175e21490a9a392d5c70ebaf46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:42.338148 2684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d39d505fc384164941d9413c9026e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.338248 3103 tablet_replica.cc:330] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: stopping tablet replica
I20260619 12:04:42.338442 3103 raft_consensus.cc:2229] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:42.338580 3103 raft_consensus.cc:2258] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:42.339172 3103 ts_tablet_manager.cc:1905] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:42.339689 3055 tablet_bootstrap.cc:492] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: Bootstrap starting.
I20260619 12:04:42.340008 2824 tablet_service.cc:1467] Processing CreateTablet for tablet 5d39d505fc384164941d9413c9026e37 (DEFAULT_TABLE table=test-table [id=3e707f175e21490a9a392d5c70ebaf46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:42.340164 2824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d39d505fc384164941d9413c9026e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.340201 3055 tablet_bootstrap.cc:654] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.341496 3103 ts_tablet_manager.cc:1918] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:42.341626 3103 log.cc:1196] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/c1a00d3d6cf2453db4ea9f118bf21335
I20260619 12:04:42.341667 3053 tablet_bootstrap.cc:492] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: Bootstrap starting.
I20260619 12:04:42.342020 3103 ts_tablet_manager.cc:1939] T c1a00d3d6cf2453db4ea9f118bf21335 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting consensus metadata
I20260619 12:04:42.342036 3053 tablet_bootstrap.cc:654] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.342238 3104 tablet_replica.cc:330] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: stopping tablet replica
I20260619 12:04:42.342394 3104 raft_consensus.cc:2229] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:42.342429 2964 tablet_service.cc:1467] Processing CreateTablet for tablet 5d39d505fc384164941d9413c9026e37 (DEFAULT_TABLE table=test-table [id=3e707f175e21490a9a392d5c70ebaf46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:42.337988 3102 ts_tablet_manager.cc:1905] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:42.342572 2964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d39d505fc384164941d9413c9026e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:42.342675 3104 raft_consensus.cc:2258] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:42.344151 3054 tablet_bootstrap.cc:492] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: Bootstrap starting.
I20260619 12:04:42.344596 3054 tablet_bootstrap.cc:654] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:42.345115 3102 ts_tablet_manager.cc:1918] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:42.345224 3102 log.cc:1196] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/c1a00d3d6cf2453db4ea9f118bf21335
I20260619 12:04:42.345515 3102 ts_tablet_manager.cc:1939] T c1a00d3d6cf2453db4ea9f118bf21335 P 72a9500749b64e5fb78892d84479f474: Deleting consensus metadata
I20260619 12:04:42.346077 3104 ts_tablet_manager.cc:1905] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:42.346102 2533 catalog_manager.cc:4841] TS 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427): tablet c1a00d3d6cf2453db4ea9f118bf21335 (table test-table [id=1188fd6e880f42a6bef06e59a2ca553d]) successfully deleted
I20260619 12:04:42.346307 2534 catalog_manager.cc:4841] TS 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123): tablet c1a00d3d6cf2453db4ea9f118bf21335 (table test-table [id=1188fd6e880f42a6bef06e59a2ca553d]) successfully deleted
I20260619 12:04:42.348788 3104 ts_tablet_manager.cc:1918] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:42.348886 3055 tablet_bootstrap.cc:492] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: No bootstrap required, opened a new log
I20260619 12:04:42.348980 3055 ts_tablet_manager.cc:1397] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260619 12:04:42.349181 3055 raft_consensus.cc:348] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.349285 3055 raft_consensus.cc:374] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.349306 3055 raft_consensus.cc:729] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a9500749b64e5fb78892d84479f474, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.349347 3055 consensus_queue.cc:260] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.349357 3054 tablet_bootstrap.cc:492] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: No bootstrap required, opened a new log
I20260619 12:04:42.349408 3054 ts_tablet_manager.cc:1397] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 12:04:42.349434 3055 ts_tablet_manager.cc:1428] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 12:04:42.349578 3054 raft_consensus.cc:348] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.349659 3054 raft_consensus.cc:374] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.349680 3054 raft_consensus.cc:729] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba3dbb406e45309472fa7fef6e2a72, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.349728 3054 consensus_queue.cc:260] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.349824 3054 ts_tablet_manager.cc:1428] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:04:42.350342 3104 log.cc:1196] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/c1a00d3d6cf2453db4ea9f118bf21335
I20260619 12:04:42.350724 3104 ts_tablet_manager.cc:1939] T c1a00d3d6cf2453db4ea9f118bf21335 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting consensus metadata
I20260619 12:04:42.351286 2531 catalog_manager.cc:4841] TS 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627): tablet c1a00d3d6cf2453db4ea9f118bf21335 (table test-table [id=1188fd6e880f42a6bef06e59a2ca553d]) successfully deleted
I20260619 12:04:42.351593 3053 tablet_bootstrap.cc:492] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: No bootstrap required, opened a new log
I20260619 12:04:42.351662 3053 ts_tablet_manager.cc:1397] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20260619 12:04:42.351823 3053 raft_consensus.cc:348] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.351910 3053 raft_consensus.cc:374] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:42.351933 3053 raft_consensus.cc:729] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd8adc6b7bd45508bc153ce2c3b9504, State: Initialized, Role: FOLLOWER
I20260619 12:04:42.351977 3053 consensus_queue.cc:260] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.352074 3053 ts_tablet_manager.cc:1428] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 12:04:42.365631 3060 raft_consensus.cc:482] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:42.365752 3060 raft_consensus.cc:504] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.366662 3060 leader_election.cc:288] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627)
I20260619 12:04:42.371747 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.366926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33427 (local address 127.0.203.131:44685)
0619 12:04:42.367260 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.367274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.367329 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.367927 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.367928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.367931 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:42.368017 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.368019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.369019 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.369020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.370324 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.370327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.371274 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.371276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.371515 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.371521 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.371575 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.371642 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309}
I20260619 12:04:42.372074 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.367682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:44685 (local address 127.0.203.129:33427)
0619 12:04:42.367733 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.367734 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.367743 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:04:42.367778 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.367779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.367790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.367866 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:42.368107 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.368867 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.370463 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.371115 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.371210 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.371221 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.371955 (+ 734us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.372009 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:04:42.372287 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d39d505fc384164941d9413c9026e37" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474" is_pre_election: true
I20260619 12:04:42.372378 2704 raft_consensus.cc:2454] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 0.
I20260619 12:04:42.375123 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.367537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:45169 (local address 127.0.203.130:35627)
0619 12:04:42.367587 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.367589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.370732 (+ 3143us) server_negotiation.cc:407] Connection header received
0619 12:04:42.370837 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.370839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.370854 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.370882 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:42.371515 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.372201 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.374115 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.374668 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.374721 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.374734 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.374968 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.375015 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:04:42.375169 2917 leader_election.cc:302] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba3dbb406e45309472fa7fef6e2a72, 72a9500749b64e5fb78892d84479f474; no voters:
I20260619 12:04:42.375391 3060 raft_consensus.cc:2781] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:42.375437 3060 raft_consensus.cc:482] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:42.375458 3060 raft_consensus.cc:3037] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.375801 3108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.367063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35627 (local address 127.0.203.131:45169)
0619 12:04:42.370655 (+ 3592us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.370668 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.370762 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.370945 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.370946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.370948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:42.371411 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.371415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.372825 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.372826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.373990 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.373993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.374797 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.374798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.374884 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.374889 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.375640 (+ 751us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.375700 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3557,"thread_start_us":73,"threads_started":1}
I20260619 12:04:42.376251 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d39d505fc384164941d9413c9026e37" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" is_pre_election: true
I20260619 12:04:42.376361 2844 raft_consensus.cc:2454] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 0.
I20260619 12:04:42.376493 3060 raft_consensus.cc:504] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.376679 3060 leader_election.cc:288] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 election: Requested vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627)
I20260619 12:04:42.376868 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d39d505fc384164941d9413c9026e37" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504"
I20260619 12:04:42.376945 2844 raft_consensus.cc:3037] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.376935 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d39d505fc384164941d9413c9026e37" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474"
I20260619 12:04:42.377023 2704 raft_consensus.cc:3037] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:42.377964 2844 raft_consensus.cc:2454] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 1.
I20260619 12:04:42.378224 2915 leader_election.cc:302] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba3dbb406e45309472fa7fef6e2a72, 9cd8adc6b7bd45508bc153ce2c3b9504; no voters:
I20260619 12:04:42.378345 2704 raft_consensus.cc:2454] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 1.
I20260619 12:04:42.378376 3060 raft_consensus.cc:2781] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:42.378511 3060 raft_consensus.cc:686] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 LEADER]: Becoming Leader. State: Replica: 0aba3dbb406e45309472fa7fef6e2a72, State: Running, Role: LEADER
I20260619 12:04:42.378579 3060 consensus_queue.cc:237] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:42.379688 2546 catalog_manager.cc:5495] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "0aba3dbb406e45309472fa7fef6e2a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } health_report { overall_health: HEALTHY } } }
I20260619 12:04:42.392789 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.389492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42388 (local address 127.0.203.131:36123)
0619 12:04:42.389532 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.389534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.389634 (+ 100us) server_negotiation.cc:407] Connection header received
0619 12:04:42.389677 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.389678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.389701 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.389764 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:42.389932 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.390687 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.391720 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.392002 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.392068 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.392269 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.392410 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.392473 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.392684 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.392707 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:04:42.399435 3036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.395771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35627 (local address 127.0.203.131:39549)
0619 12:04:42.395902 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.395914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.395968 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.396251 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.396252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.396253 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:42.396319 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.396321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.397209 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.397211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.398374 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.398380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.398917 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.398919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.398978 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.399009 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.399202 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.399204 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.399240 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.399292 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260619 12:04:42.399751 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.396069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:39549 (local address 127.0.203.130:35627)
0619 12:04:42.396114 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.396115 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.396122 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:04:42.396148 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.396149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.396159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.396204 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:42.396395 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.397116 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.398536 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.398798 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.398854 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.399106 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.399144 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.399182 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.399653 (+ 471us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.399677 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:04:42.402655 2844 raft_consensus.cc:1261] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Refusing update from remote peer 0aba3dbb406e45309472fa7fef6e2a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:42.402979 2704 raft_consensus.cc:1261] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Refusing update from remote peer 0aba3dbb406e45309472fa7fef6e2a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:42.403172 3109 consensus_queue.cc:1035] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:42.403299 3060 consensus_queue.cc:1035] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:42.414384 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.411056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:36123 (local address 127.0.203.130:41319)
0619 12:04:42.411136 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:42.411157 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:42.411279 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:42.411524 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:42.411525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:42.411527 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:42.411591 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.411594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.412500 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.412501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.413310 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:42.413313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.413928 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.413930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:42.414005 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.414016 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.414190 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.414192 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:04:42.414230 (+ 38us) client_negotiation.cc:240] Negotiation successful
0619 12:04:42.414297 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 12:04:42.414662 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:42.411147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:41319 (local address 127.0.203.131:36123)
0619 12:04:42.411202 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:42.411204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:42.411255 (+ 51us) server_negotiation.cc:407] Connection header received
0619 12:04:42.411405 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:42.411407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:42.411422 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:42.411465 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:42.411666 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.412398 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.413444 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:42.413800 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:42.413879 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:42.414090 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:42.414130 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:42.414169 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:42.414589 (+ 420us) server_negotiation.cc:299] Negotiation successful
0619 12:04:42.414612 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 12:04:42.414945 3084 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 12:04:43.944926 2547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:43.945094 2547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:43.946094 2547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Sending DeleteTablet for 3 replicas of tablet 5d39d505fc384164941d9413c9026e37
I20260619 12:04:43.946405 2824 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d39d505fc384164941d9413c9026e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:43 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:40702
I20260619 12:04:43.946435 2964 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d39d505fc384164941d9413c9026e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:43 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42382
I20260619 12:04:43.946431 2684 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d39d505fc384164941d9413c9026e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:43 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:32954
I20260619 12:04:43.946692 3143 tablet_replica.cc:330] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: stopping tablet replica
I20260619 12:04:43.946755 3143 raft_consensus.cc:2229] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:04:43.946884 3143 raft_consensus.cc:2258] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:43.947001 3144 tablet_replica.cc:330] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: stopping tablet replica
I20260619 12:04:43.947059 3144 raft_consensus.cc:2229] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:43.947113 3144 raft_consensus.cc:2258] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:43.947393 3143 ts_tablet_manager.cc:1905] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:43.949519 3143 ts_tablet_manager.cc:1918] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:04:43.949572 3143 log.cc:1196] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/5d39d505fc384164941d9413c9026e37
I20260619 12:04:43.949785 3143 ts_tablet_manager.cc:1939] T 5d39d505fc384164941d9413c9026e37 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting consensus metadata
I20260619 12:04:43.950119 3142 tablet_replica.cc:330] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: stopping tablet replica
I20260619 12:04:43.950201 3142 raft_consensus.cc:2229] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:43.950255 3142 raft_consensus.cc:2258] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:43.950233 2534 catalog_manager.cc:4841] TS 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123): tablet 5d39d505fc384164941d9413c9026e37 (table test-table [id=3e707f175e21490a9a392d5c70ebaf46]) successfully deleted
I20260619 12:04:43.951668 3142 ts_tablet_manager.cc:1905] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:43.953320 3142 ts_tablet_manager.cc:1918] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:04:43.953387 3142 log.cc:1196] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/5d39d505fc384164941d9413c9026e37
I20260619 12:04:43.953579 3142 ts_tablet_manager.cc:1939] T 5d39d505fc384164941d9413c9026e37 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting consensus metadata
I20260619 12:04:43.953953 2531 catalog_manager.cc:4841] TS 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627): tablet 5d39d505fc384164941d9413c9026e37 (table test-table [id=3e707f175e21490a9a392d5c70ebaf46]) successfully deleted
I20260619 12:04:43.953981 3144 ts_tablet_manager.cc:1905] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:43.955130 3144 ts_tablet_manager.cc:1918] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:04:43.955184 3144 log.cc:1196] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/5d39d505fc384164941d9413c9026e37
I20260619 12:04:43.955410 3144 ts_tablet_manager.cc:1939] T 5d39d505fc384164941d9413c9026e37 P 72a9500749b64e5fb78892d84479f474: Deleting consensus metadata
I20260619 12:04:43.955845 2533 catalog_manager.cc:4841] TS 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427): tablet 5d39d505fc384164941d9413c9026e37 (table test-table [id=3e707f175e21490a9a392d5c70ebaf46]) successfully deleted
Jun 19 12:04:43 dist-test-slave-44vj krb5kdc[2502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:04:43 dist-test-slave-44vj krb5kdc[2502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:43.979809 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:43.965804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.0.203.190:34681)
0619 12:04:43.965969 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:43.965971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:43.965983 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:04:43.966024 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:43.966025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:43.966038 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:43.966122 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:43.966723 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:43.967309 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:43.968117 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:43.968322 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:43.968380 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:43.976894 (+ 8514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:43.976898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:43.976898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:43.976911 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:43.978483 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:43.978928 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:43.978929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:43.978930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:43.978993 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:43.979231 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:43.979231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:43.979231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:43.979405 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:43.979500 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:43.979620 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 12:04:43.979719 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260619 12:04:43.981789 2547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:43.981999 2547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:43.983763 2824 tablet_service.cc:1467] Processing CreateTablet for tablet fa6e020d50d7410b815b9a8987a833bd (DEFAULT_TABLE table=test-table [id=023a6ba962cf4784b3d84b20986da61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:43.983793 2964 tablet_service.cc:1467] Processing CreateTablet for tablet fa6e020d50d7410b815b9a8987a833bd (DEFAULT_TABLE table=test-table [id=023a6ba962cf4784b3d84b20986da61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:43.983848 2684 tablet_service.cc:1467] Processing CreateTablet for tablet fa6e020d50d7410b815b9a8987a833bd (DEFAULT_TABLE table=test-table [id=023a6ba962cf4784b3d84b20986da61e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:43.983958 2824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa6e020d50d7410b815b9a8987a833bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:43.983960 2964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa6e020d50d7410b815b9a8987a833bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:43.983976 2684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa6e020d50d7410b815b9a8987a833bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:43.985663 3155 tablet_bootstrap.cc:492] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: Bootstrap starting.
I20260619 12:04:43.985699 3154 tablet_bootstrap.cc:492] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: Bootstrap starting.
I20260619 12:04:43.986187 3155 tablet_bootstrap.cc:654] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:43.986187 3154 tablet_bootstrap.cc:654] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:43.986585 3153 tablet_bootstrap.cc:492] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: Bootstrap starting.
I20260619 12:04:43.987025 3155 tablet_bootstrap.cc:492] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: No bootstrap required, opened a new log
I20260619 12:04:43.987048 3153 tablet_bootstrap.cc:654] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:43.987135 3155 ts_tablet_manager.cc:1397] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:04:43.987289 3155 raft_consensus.cc:348] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:43.987454 3155 raft_consensus.cc:374] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:43.987526 3155 raft_consensus.cc:729] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd8adc6b7bd45508bc153ce2c3b9504, State: Initialized, Role: FOLLOWER
I20260619 12:04:43.987593 3155 consensus_queue.cc:260] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:43.987713 3155 ts_tablet_manager.cc:1428] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:04:43.987769 3153 tablet_bootstrap.cc:492] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: No bootstrap required, opened a new log
I20260619 12:04:43.987823 3153 ts_tablet_manager.cc:1397] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:43.988003 3153 raft_consensus.cc:348] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:43.988116 3153 raft_consensus.cc:374] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:43.988137 3153 raft_consensus.cc:729] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a9500749b64e5fb78892d84479f474, State: Initialized, Role: FOLLOWER
I20260619 12:04:43.987562 3154 tablet_bootstrap.cc:492] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: No bootstrap required, opened a new log
I20260619 12:04:43.988257 3154 ts_tablet_manager.cc:1397] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:04:43.988406 3154 raft_consensus.cc:348] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:43.988472 3154 raft_consensus.cc:374] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:43.988488 3154 raft_consensus.cc:729] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba3dbb406e45309472fa7fef6e2a72, State: Initialized, Role: FOLLOWER
I20260619 12:04:43.988566 3153 consensus_queue.cc:260] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:43.988793 3153 ts_tablet_manager.cc:1428] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:43.988885 3154 consensus_queue.cc:260] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:43.989035 3154 ts_tablet_manager.cc:1428] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:04:44.364331 3136 raft_consensus.cc:482] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:44.364457 3136 raft_consensus.cc:504] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.364682 3136 leader_election.cc:288] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627)
I20260619 12:04:44.364974 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa6e020d50d7410b815b9a8987a833bd" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" is_pre_election: true
I20260619 12:04:44.364977 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa6e020d50d7410b815b9a8987a833bd" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474" is_pre_election: true
I20260619 12:04:44.365079 2844 raft_consensus.cc:2454] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 0.
I20260619 12:04:44.365104 2704 raft_consensus.cc:2454] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 0.
I20260619 12:04:44.365315 2915 leader_election.cc:302] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba3dbb406e45309472fa7fef6e2a72, 9cd8adc6b7bd45508bc153ce2c3b9504; no voters:
I20260619 12:04:44.365463 3136 raft_consensus.cc:2781] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:44.365535 3136 raft_consensus.cc:482] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:44.365549 3136 raft_consensus.cc:3037] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:44.366374 3136 raft_consensus.cc:504] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.366534 3136 leader_election.cc:288] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 election: Requested vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627)
I20260619 12:04:44.366779 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa6e020d50d7410b815b9a8987a833bd" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474"
I20260619 12:04:44.366853 2704 raft_consensus.cc:3037] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:44.367456 2704 raft_consensus.cc:2454] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 1.
I20260619 12:04:44.367601 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa6e020d50d7410b815b9a8987a833bd" candidate_uuid: "0aba3dbb406e45309472fa7fef6e2a72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504"
I20260619 12:04:44.367702 2844 raft_consensus.cc:3037] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:44.367698 2917 leader_election.cc:302] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba3dbb406e45309472fa7fef6e2a72, 72a9500749b64e5fb78892d84479f474; no voters:
I20260619 12:04:44.367821 3136 raft_consensus.cc:2781] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:44.367893 3136 raft_consensus.cc:686] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 LEADER]: Becoming Leader. State: Replica: 0aba3dbb406e45309472fa7fef6e2a72, State: Running, Role: LEADER
I20260619 12:04:44.367956 3136 consensus_queue.cc:237] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.368410 2844 raft_consensus.cc:2454] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba3dbb406e45309472fa7fef6e2a72 in term 1.
I20260619 12:04:44.368736 2547 catalog_manager.cc:5495] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "0aba3dbb406e45309472fa7fef6e2a72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } health_report { overall_health: HEALTHY } } }
I20260619 12:04:44.432994 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:44.430515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.0.203.131:36123)
0619 12:04:44.430676 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:44.430679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:44.430712 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:04:44.430783 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:44.430783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:44.430795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:44.430864 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:44.430999 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.431668 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.432444 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.432641 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.432690 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:44.432789 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:44.432818 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:44.432854 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:44.432892 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 12:04:44.432911 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260619 12:04:44.435648 2844 raft_consensus.cc:1261] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Refusing update from remote peer 0aba3dbb406e45309472fa7fef6e2a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:44.435701 2704 raft_consensus.cc:1261] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Refusing update from remote peer 0aba3dbb406e45309472fa7fef6e2a72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:44.435907 3136 consensus_queue.cc:1035] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:44.436026 3120 consensus_queue.cc:1035] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:44.447078 2547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:44.447213 2547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:44.448062 2547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Sending DeleteTablet for 3 replicas of tablet fa6e020d50d7410b815b9a8987a833bd
I20260619 12:04:44.448369 2824 tablet_service.cc:1514] Processing DeleteTablet for tablet fa6e020d50d7410b815b9a8987a833bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:44 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:40702
I20260619 12:04:44.448378 2964 tablet_service.cc:1514] Processing DeleteTablet for tablet fa6e020d50d7410b815b9a8987a833bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:44 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42382
I20260619 12:04:44.448521 3143 tablet_replica.cc:330] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: stopping tablet replica
I20260619 12:04:44.448515 2684 tablet_service.cc:1514] Processing DeleteTablet for tablet fa6e020d50d7410b815b9a8987a833bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:44 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:32954
I20260619 12:04:44.448596 3143 raft_consensus.cc:2229] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:04:44.448621 3142 tablet_replica.cc:330] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: stopping tablet replica
I20260619 12:04:44.448673 3143 raft_consensus.cc:2258] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:44.448686 3142 raft_consensus.cc:2229] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:44.448733 3142 raft_consensus.cc:2258] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:44.448802 3144 tablet_replica.cc:330] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: stopping tablet replica
I20260619 12:04:44.448868 3144 raft_consensus.cc:2229] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:44.448912 3144 raft_consensus.cc:2258] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:44.449028 3143 ts_tablet_manager.cc:1905] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:44.449276 3142 ts_tablet_manager.cc:1905] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:44.449448 2547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:44.449646 2547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:44.450850 2684 tablet_service.cc:1467] Processing CreateTablet for tablet df0bbb8889964c88b9d0cace6943d0c0 (DEFAULT_TABLE table=test-table [id=3dc91e46c112417fa2e36846a2271537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:44.450946 3142 ts_tablet_manager.cc:1918] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:44.450984 2684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df0bbb8889964c88b9d0cace6943d0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:44.451004 3142 log.cc:1196] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/fa6e020d50d7410b815b9a8987a833bd
I20260619 12:04:44.451025 3143 ts_tablet_manager.cc:1918] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:44.451064 3143 log.cc:1196] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/fa6e020d50d7410b815b9a8987a833bd
I20260619 12:04:44.451268 3143 ts_tablet_manager.cc:1939] T fa6e020d50d7410b815b9a8987a833bd P 0aba3dbb406e45309472fa7fef6e2a72: Deleting consensus metadata
I20260619 12:04:44.451268 3142 ts_tablet_manager.cc:1939] T fa6e020d50d7410b815b9a8987a833bd P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting consensus metadata
I20260619 12:04:44.451644 3144 ts_tablet_manager.cc:1905] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:44.452404 3153 tablet_bootstrap.cc:492] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474: Bootstrap starting.
I20260619 12:04:44.452847 3153 tablet_bootstrap.cc:654] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:44.452822 2824 tablet_service.cc:1467] Processing CreateTablet for tablet df0bbb8889964c88b9d0cace6943d0c0 (DEFAULT_TABLE table=test-table [id=3dc91e46c112417fa2e36846a2271537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:44.452951 2824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df0bbb8889964c88b9d0cace6943d0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:44.453176 3144 ts_tablet_manager.cc:1918] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:04:44.453243 3144 log.cc:1196] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/fa6e020d50d7410b815b9a8987a833bd
I20260619 12:04:44.453455 3144 ts_tablet_manager.cc:1939] T fa6e020d50d7410b815b9a8987a833bd P 72a9500749b64e5fb78892d84479f474: Deleting consensus metadata
I20260619 12:04:44.453500 3153 tablet_bootstrap.cc:492] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474: No bootstrap required, opened a new log
I20260619 12:04:44.453547 3153 ts_tablet_manager.cc:1397] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:44.453692 3153 raft_consensus.cc:348] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.453783 3153 raft_consensus.cc:374] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:44.453802 3153 raft_consensus.cc:729] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a9500749b64e5fb78892d84479f474, State: Initialized, Role: FOLLOWER
I20260619 12:04:44.453843 3153 consensus_queue.cc:260] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.453953 3153 ts_tablet_manager.cc:1428] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:44.454270 2531 catalog_manager.cc:4841] TS 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627): tablet fa6e020d50d7410b815b9a8987a833bd (table test-table [id=023a6ba962cf4784b3d84b20986da61e]) successfully deleted
I20260619 12:04:44.454413 2533 catalog_manager.cc:4841] TS 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427): tablet fa6e020d50d7410b815b9a8987a833bd (table test-table [id=023a6ba962cf4784b3d84b20986da61e]) successfully deleted
I20260619 12:04:44.454665 2534 catalog_manager.cc:4841] TS 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123): tablet fa6e020d50d7410b815b9a8987a833bd (table test-table [id=023a6ba962cf4784b3d84b20986da61e]) successfully deleted
I20260619 12:04:44.454741 3155 tablet_bootstrap.cc:492] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: Bootstrap starting.
I20260619 12:04:44.454752 2964 tablet_service.cc:1467] Processing CreateTablet for tablet df0bbb8889964c88b9d0cace6943d0c0 (DEFAULT_TABLE table=test-table [id=3dc91e46c112417fa2e36846a2271537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:44.454862 2964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df0bbb8889964c88b9d0cace6943d0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:44.455101 3155 tablet_bootstrap.cc:654] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:44.455896 3155 tablet_bootstrap.cc:492] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: No bootstrap required, opened a new log
I20260619 12:04:44.455940 3155 ts_tablet_manager.cc:1397] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:44.456101 3155 raft_consensus.cc:348] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.456175 3155 raft_consensus.cc:374] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:44.456194 3155 raft_consensus.cc:729] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd8adc6b7bd45508bc153ce2c3b9504, State: Initialized, Role: FOLLOWER
I20260619 12:04:44.456250 3155 consensus_queue.cc:260] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.456353 3154 tablet_bootstrap.cc:492] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: Bootstrap starting.
I20260619 12:04:44.456442 3155 ts_tablet_manager.cc:1428] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:04:44.456777 3154 tablet_bootstrap.cc:654] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:44.457383 3154 tablet_bootstrap.cc:492] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: No bootstrap required, opened a new log
I20260619 12:04:44.457432 3154 ts_tablet_manager.cc:1397] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:44.457557 3154 raft_consensus.cc:348] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.457633 3154 raft_consensus.cc:374] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:44.457646 3154 raft_consensus.cc:729] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba3dbb406e45309472fa7fef6e2a72, State: Initialized, Role: FOLLOWER
I20260619 12:04:44.457690 3154 consensus_queue.cc:260] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.457792 3154 ts_tablet_manager.cc:1428] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:44.642161 3166 raft_consensus.cc:482] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:44.642298 3166 raft_consensus.cc:504] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.642503 3166 leader_election.cc:288] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123)
I20260619 12:04:44.642813 2984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0bbb8889964c88b9d0cace6943d0c0" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba3dbb406e45309472fa7fef6e2a72" is_pre_election: true
I20260619 12:04:44.642917 2984 raft_consensus.cc:2454] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 0.
I20260619 12:04:44.642931 2703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0bbb8889964c88b9d0cace6943d0c0" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474" is_pre_election: true
I20260619 12:04:44.643100 2703 raft_consensus.cc:2454] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 0.
I20260619 12:04:44.643232 2778 leader_election.cc:302] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba3dbb406e45309472fa7fef6e2a72, 9cd8adc6b7bd45508bc153ce2c3b9504; no voters:
I20260619 12:04:44.643344 3166 raft_consensus.cc:2781] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:44.643428 3166 raft_consensus.cc:482] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:44.643445 3166 raft_consensus.cc:3037] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:44.644219 3166 raft_consensus.cc:504] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.644368 3166 leader_election.cc:288] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 election: Requested vote from peers 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427), 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123)
I20260619 12:04:44.644557 2984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0bbb8889964c88b9d0cace6943d0c0" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba3dbb406e45309472fa7fef6e2a72"
I20260619 12:04:44.644543 2703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0bbb8889964c88b9d0cace6943d0c0" candidate_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a9500749b64e5fb78892d84479f474"
I20260619 12:04:44.644636 2984 raft_consensus.cc:3037] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:44.644637 2703 raft_consensus.cc:3037] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:44.645390 2984 raft_consensus.cc:2454] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 1.
I20260619 12:04:44.645406 2703 raft_consensus.cc:2454] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd8adc6b7bd45508bc153ce2c3b9504 in term 1.
I20260619 12:04:44.645613 2777 leader_election.cc:302] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72a9500749b64e5fb78892d84479f474, 9cd8adc6b7bd45508bc153ce2c3b9504; no voters:
I20260619 12:04:44.645805 3166 raft_consensus.cc:2781] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:44.645866 3166 raft_consensus.cc:686] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 LEADER]: Becoming Leader. State: Replica: 9cd8adc6b7bd45508bc153ce2c3b9504, State: Running, Role: LEADER
I20260619 12:04:44.645926 3166 consensus_queue.cc:237] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } }
I20260619 12:04:44.646597 2547 catalog_manager.cc:5495] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130). New cstate: current_term: 1 leader_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cd8adc6b7bd45508bc153ce2c3b9504" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 } health_report { overall_health: UNKNOWN } } }
I20260619 12:04:44.690814 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:44.688489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.0.203.130:35627)
0619 12:04:44.688699 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:44.688701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:44.688706 (+ 5us) server_negotiation.cc:407] Connection header received
0619 12:04:44.688725 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:44.688725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:44.688735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:44.688793 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:44.688941 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.689444 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.690152 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.690368 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.690421 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:44.690552 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:44.690588 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:44.690629 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:44.690681 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 12:04:44.690703 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":122,"threads_started":1}
I20260619 12:04:44.695909 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:44.693574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:41093 (local address 127.0.203.130:35627)
0619 12:04:44.693609 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:44.693611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:44.693667 (+ 56us) server_negotiation.cc:407] Connection header received
0619 12:04:44.693732 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:44.693733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:44.693742 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:44.693792 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:44.693917 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.694362 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.695156 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.695425 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.695496 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:44.695623 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:44.695650 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:44.695706 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:44.695755 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 12:04:44.695775 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 12:04:44.695935 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:44.693455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35627 (local address 127.0.203.130:41093)
0619 12:04:44.693601 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:44.693613 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:44.693683 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:44.693806 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:44.693807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:44.693809 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:04:44.693868 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:44.693870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.694440 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.694441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:44.695080 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:44.695082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:44.695507 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:44.695508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:44.695563 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:44.695568 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:04:44.695708 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:04:44.695709 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:04:44.695739 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 12:04:44.695814 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260619 12:04:44.698261 2703 raft_consensus.cc:1261] T df0bbb8889964c88b9d0cace6943d0c0 P 72a9500749b64e5fb78892d84479f474 [term 1 FOLLOWER]: Refusing update from remote peer 9cd8adc6b7bd45508bc153ce2c3b9504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:44.698457 2984 raft_consensus.cc:1261] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Refusing update from remote peer 9cd8adc6b7bd45508bc153ce2c3b9504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:44.698550 3119 consensus_queue.cc:1035] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a9500749b64e5fb78892d84479f474" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:44.698721 3166 consensus_queue.cc:1035] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0aba3dbb406e45309472fa7fef6e2a72" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 36123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:04:44.706182 3121 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 12:04:46.228173 2547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:46.228331 2547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:46.229498 2547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10795a945a964778aeccd477229a7bb4: Sending DeleteTablet for 3 replicas of tablet df0bbb8889964c88b9d0cace6943d0c0
I20260619 12:04:46.229830 2684 tablet_service.cc:1514] Processing DeleteTablet for tablet df0bbb8889964c88b9d0cace6943d0c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:46 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:32954
I20260619 12:04:46.229867 2824 tablet_service.cc:1514] Processing DeleteTablet for tablet df0bbb8889964c88b9d0cace6943d0c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:46 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:40702
I20260619 12:04:46.230011 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 2619
I20260619 12:04:46.230137 2964 tablet_service.cc:1514] Processing DeleteTablet for tablet df0bbb8889964c88b9d0cace6943d0c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:46 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42382
I20260619 12:04:46.230921 3199 tablet_replica.cc:330] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: stopping tablet replica
I20260619 12:04:46.230993 3199 raft_consensus.cc:2229] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:46.231045 3199 raft_consensus.cc:2258] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:46.231334 3199 ts_tablet_manager.cc:1905] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:46.233243 3199 ts_tablet_manager.cc:1918] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:04:46.233315 3199 log.cc:1196] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/df0bbb8889964c88b9d0cace6943d0c0
I20260619 12:04:46.233649 3199 ts_tablet_manager.cc:1939] T df0bbb8889964c88b9d0cace6943d0c0 P 0aba3dbb406e45309472fa7fef6e2a72: Deleting consensus metadata
I20260619 12:04:46.234455 2534 catalog_manager.cc:4841] TS 0aba3dbb406e45309472fa7fef6e2a72 (127.0.203.131:36123): tablet df0bbb8889964c88b9d0cace6943d0c0 (table test-table [id=3dc91e46c112417fa2e36846a2271537]) successfully deleted
I20260619 12:04:46.235553 3198 tablet_replica.cc:330] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: stopping tablet replica
I20260619 12:04:46.235687 3198 raft_consensus.cc:2229] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:04:46.235877 3198 raft_consensus.cc:2258] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:46.236315 3198 ts_tablet_manager.cc:1905] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:46.238123 3198 ts_tablet_manager.cc:1918] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:04:46.238199 3198 log.cc:1196] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/df0bbb8889964c88b9d0cace6943d0c0
I20260619 12:04:46.238427 3198 ts_tablet_manager.cc:1939] T df0bbb8889964c88b9d0cace6943d0c0 P 9cd8adc6b7bd45508bc153ce2c3b9504: Deleting consensus metadata
I20260619 12:04:46.238878 2531 catalog_manager.cc:4841] TS 9cd8adc6b7bd45508bc153ce2c3b9504 (127.0.203.130:35627): tablet df0bbb8889964c88b9d0cace6943d0c0 (table test-table [id=3dc91e46c112417fa2e36846a2271537]) successfully deleted
W20260619 12:04:46.239001 2533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:33427 (error 108)
I20260619 12:04:46.239204 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 2759
I20260619 12:04:46.240767 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:46.240532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33427 (local address 127.0.0.1:39578)
0619 12:04:46.240673 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:46.240721 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:33427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
W20260619 12:04:46.240965 2533 catalog_manager.cc:4584] TS 72a9500749b64e5fb78892d84479f474 (127.0.203.129:33427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df0bbb8889964c88b9d0cace6943d0c0: Network error: Client connection negotiation failed: client connection to 127.0.203.129:33427: connect: Connection refused (error 111)
I20260619 12:04:46.245416 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 2899
I20260619 12:04:46.251125 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 2515
2026-06-19T12:04:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6113 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:46 dist-test-slave-44vj krb5kdc[3204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:46 dist-test-slave-44vj krb5kdc[3204](info): set up 2 sockets
Jun 19 12:04:46 dist-test-slave-44vj krb5kdc[3204](info): commencing operation
krb5kdc: starting...
W20260619 12:04:47.215451 814 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:47 dist-test-slave-44vj krb5kdc[3204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:47.344023 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41549
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:41549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:47.446069 3217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:47.446386 3217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:47.446424 3217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:47.448200 3217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:47.448279 3217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:47.448299 3217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:47.448316 3217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:47.448333 3217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:47.451227 3217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:41549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:47.451617 3217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:47.451913 3217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:47.457327 3225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:47.457628 3223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:47.458097 3217 server_base.cc:1034] running on GCE node
W20260619 12:04:47.458676 3222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:47.459004 3217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:47.459399 3217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:47.460592 3217 hybrid_clock.cc:648] HybridClock initialized: now 1781870687460571 us; error 41 us; skew 500 ppm
Jun 19 12:04:47 dist-test-slave-44vj krb5kdc[3204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:47.463672 3217 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:47.464183 3217 webserver.cc:460] Webserver started at http://127.0.203.190:43083/ using document root <none> and password file <none>
I20260619 12:04:47.464442 3217 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:47.464494 3217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:47.464638 3217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:47.465813 3217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "4a843bf1f13846568c95818095114446"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "621ccb2202769fc03ec99f4abcaefa82"
server_key_iv: "f8b9da6743a82e04d06c8fa3550e9769"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.466154 3217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "4a843bf1f13846568c95818095114446"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "621ccb2202769fc03ec99f4abcaefa82"
server_key_iv: "f8b9da6743a82e04d06c8fa3550e9769"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.467595 3217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:47.468330 3232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:47.468518 3217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:47.468614 3217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "4a843bf1f13846568c95818095114446"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "621ccb2202769fc03ec99f4abcaefa82"
server_key_iv: "f8b9da6743a82e04d06c8fa3550e9769"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.468677 3217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:47.490962 3217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:47.493237 3217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:47.493386 3217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:47.497736 3217 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:41549
I20260619 12:04:47.497822 3284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:41549 every 8 connection(s)
I20260619 12:04:47.498466 3217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:47.499213 3285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:47.499578 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 3217
I20260619 12:04:47.499673 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:47.499872 814 external_mini_cluster.cc:1428] Setting key 4836e108285cb5ea14e3b5609684d0a8
I20260619 12:04:47.502311 3285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446: Bootstrap starting.
I20260619 12:04:47.502961 3285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:47.503212 3285 log.cc:824] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:47.503918 3285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446: No bootstrap required, opened a new log
I20260619 12:04:47.505800 3285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a843bf1f13846568c95818095114446" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41549 } }
I20260619 12:04:47.505962 3285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:47.505996 3285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a843bf1f13846568c95818095114446, State: Initialized, Role: FOLLOWER
I20260619 12:04:47.506131 3285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a843bf1f13846568c95818095114446" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41549 } }
I20260619 12:04:47.506206 3285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:47.506237 3285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:47.506284 3285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:47.506856 3285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a843bf1f13846568c95818095114446" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41549 } }
Jun 19 12:04:47 dist-test-slave-44vj krb5kdc[3204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:47.506958 3285 leader_election.cc:302] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a843bf1f13846568c95818095114446; no voters:
I20260619 12:04:47.507129 3285 leader_election.cc:288] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:47.507218 3290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:47.507475 3285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:47.507470 3290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [term 1 LEADER]: Becoming Leader. State: Replica: 4a843bf1f13846568c95818095114446, State: Running, Role: LEADER
I20260619 12:04:47.509016 3290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a843bf1f13846568c95818095114446" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41549 } }
W20260619 12:04:47.510422 3304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:04:47.510479 3304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:04:47.509829 3292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a843bf1f13846568c95818095114446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a843bf1f13846568c95818095114446" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41549 } } }
I20260619 12:04:47.510653 3292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:47.509997 3291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a843bf1f13846568c95818095114446. Latest consensus state: current_term: 1 leader_uuid: "4a843bf1f13846568c95818095114446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a843bf1f13846568c95818095114446" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41549 } } }
I20260619 12:04:47.510875 3291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:47.510921 3307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:47.511096 3307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:47.512097 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:47.500544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.0.203.190:41549)
0619 12:04:47.500791 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:47.500805 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:47.500873 (+ 68us) server_negotiation.cc:407] Connection header received
0619 12:04:47.501493 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:47.501503 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:47.501570 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:47.501719 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:47.502424 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.503544 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.504343 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.504821 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.504926 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:47.507595 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:47.507600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:47.507600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:47.507614 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:47.510069 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:47.510612 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:47.510615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:47.510616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:47.510722 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:47.510977 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:47.510978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:47.510979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:47.511333 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:47.511580 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:47.511710 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 12:04:47.511871 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260619 12:04:47.513060 3307 catalog_manager.cc:1347] Generated new cluster ID: 3ec9577e4dfa48db804056dd37c24252
I20260619 12:04:47.513113 3307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:47.530624 3307 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:47.531546 3307 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:47.545980 3307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a843bf1f13846568c95818095114446: Generated new TSK 0
I20260619 12:04:47.546231 3307 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:47.579038 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41549
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:47.669320 3311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:47.669600 3311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:47.669626 3311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:47.670974 3311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:47.671028 3311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:47.671056 3311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:47.672895 3311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:47.673170 3311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:47.673413 3311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:47.679795 3317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:47.679795 3319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:47.680056 3311 server_base.cc:1034] running on GCE node
W20260619 12:04:47.679795 3316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:47.680395 3311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:47.680720 3311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:47.681881 3311 hybrid_clock.cc:648] HybridClock initialized: now 1781870687681866 us; error 44 us; skew 500 ppm
Jun 19 12:04:47 dist-test-slave-44vj krb5kdc[3204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:47.685230 3311 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:47.685827 3311 webserver.cc:460] Webserver started at http://127.0.203.129:44053/ using document root <none> and password file <none>
I20260619 12:04:47.686103 3311 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:47.686159 3311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:47.686306 3311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:47.687687 3311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "fd65a231f6404326a14d7618b51a71be"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "1b1cb32390e0e7df0fe5fb211b6b1273"
server_key_iv: "edf6e944b22ecf07c1bef936bebe2e05"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.688113 3311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "fd65a231f6404326a14d7618b51a71be"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "1b1cb32390e0e7df0fe5fb211b6b1273"
server_key_iv: "edf6e944b22ecf07c1bef936bebe2e05"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.690091 3311 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:04:47.691154 3326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:47.691480 3311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:47.691601 3311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "fd65a231f6404326a14d7618b51a71be"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "1b1cb32390e0e7df0fe5fb211b6b1273"
server_key_iv: "edf6e944b22ecf07c1bef936bebe2e05"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.691713 3311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:47.727185 3311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:47.729938 3311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:47.730106 3311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:47.730525 3311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:47.730952 3311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:47.730996 3311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:47.731019 3311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:47.731031 3311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:47.737758 3311 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:38541
I20260619 12:04:47.737849 3439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:38541 every 8 connection(s)
I20260619 12:04:47.738190 3311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:04:47 dist-test-slave-44vj krb5kdc[3204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:47.745788 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 3311
I20260619 12:04:47.745884 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:47.746058 814 external_mini_cluster.cc:1428] Setting key 31369909bacacdf525cfd10b31413859
I20260619 12:04:47.749464 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:47.739074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:41389 (local address 127.0.203.190:41549)
0619 12:04:47.739148 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:47.739150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:47.739336 (+ 186us) server_negotiation.cc:407] Connection header received
0619 12:04:47.740363 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:47.740364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:47.740377 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:47.740469 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:47.741295 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.741900 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.743075 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.743324 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.743422 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:47.746242 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:47.746244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:47.746245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:47.746255 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:47.747905 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:47.748403 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:47.748404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:47.748404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:47.748463 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:47.748712 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:47.748713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:47.748714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:47.748848 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:47.748908 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:47.749225 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 12:04:47.749347 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:04:47.749752 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:47.739020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41549 (local address 127.0.203.129:41389)
0619 12:04:47.739189 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:47.739237 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:47.740247 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:47.740607 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:47.740612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:47.740907 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:47.741142 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:47.741147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.742049 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.742049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:47.742940 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:47.742944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.743454 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.743455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:47.743593 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:47.744227 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:47.744238 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:47.746117 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:47.747998 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:47.748011 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:47.748016 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:47.748314 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:47.748521 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:47.748522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:47.748522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:47.748655 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:47.748904 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:47.748911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:47.749088 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 12:04:47.749199 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 12:04:47.749407 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260619 12:04:47.750365 3440 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41549
I20260619 12:04:47.750470 3440 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:47.750697 3440 heartbeater.cc:511] Master 127.0.203.190:41549 requested a full tablet report, sending...
I20260619 12:04:47.751480 3249 ts_manager.cc:194] Registered new tserver with Master: fd65a231f6404326a14d7618b51a71be (127.0.203.129:38541)
I20260619 12:04:47.752177 3249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:41389
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:47.774845 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41549
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:47.863180 3445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:47.863469 3445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:47.863508 3445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:47.864800 3445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:47.864848 3445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:47.864898 3445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:47.866592 3445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:47.866899 3445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:47.867128 3445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:47.872654 3453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:47.872735 3450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:47.872654 3451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:47.872997 3445 server_base.cc:1034] running on GCE node
I20260619 12:04:47.873276 3445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:47.873580 3445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:47.874765 3445 hybrid_clock.cc:648] HybridClock initialized: now 1781870687874762 us; error 58 us; skew 500 ppm
Jun 19 12:04:47 dist-test-slave-44vj krb5kdc[3204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:47.877739 3445 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:47.878274 3445 webserver.cc:460] Webserver started at http://127.0.203.130:42251/ using document root <none> and password file <none>
I20260619 12:04:47.878516 3445 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:47.878569 3445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:47.878702 3445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:47.879880 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "84d11212d86945ac968fca3e9e99dfa4"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "66f682f11afa46df29833124c29598a1"
server_key_iv: "f44aa566932babd5bc388e22cf91a39e"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.880232 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "84d11212d86945ac968fca3e9e99dfa4"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "66f682f11afa46df29833124c29598a1"
server_key_iv: "f44aa566932babd5bc388e22cf91a39e"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.881783 3445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:47.882748 3460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:47.883023 3445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:47.883132 3445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "84d11212d86945ac968fca3e9e99dfa4"
format_stamp: "Formatted at 2026-06-19 12:04:47 on dist-test-slave-44vj"
server_key: "66f682f11afa46df29833124c29598a1"
server_key_iv: "f44aa566932babd5bc388e22cf91a39e"
server_key_version: "encryptionkey@0"
I20260619 12:04:47.883440 3445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:47.909025 3445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:47.911526 3445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:47.911773 3445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:47.912065 3445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:47.912436 3445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:47.912472 3445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:47.912506 3445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:47.912528 3445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:47.919737 3445 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:36621
I20260619 12:04:47.919816 3573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:36621 every 8 connection(s)
I20260619 12:04:47.920166 3445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:47.921193 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 3445
I20260619 12:04:47.921301 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:47.921532 814 external_mini_cluster.cc:1428] Setting key 4cdca8db30d06cf503a91b0ee8bfb28b
Jun 19 12:04:47 dist-test-slave-44vj krb5kdc[3204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
I20260619 12:04:47.932156 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:47.921063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:40585 (local address 127.0.203.190:41549)
0619 12:04:47.921154 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:47.921156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:47.921388 (+ 232us) server_negotiation.cc:407] Connection header received
0619 12:04:47.922581 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:47.922582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:47.922639 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:47.922703 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:47.923688 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.924250 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.925753 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.925956 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.926036 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:47.929037 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:47.929040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:47.929040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:47.929050 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:47.930542 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:47.931044 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:47.931045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:47.931046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:47.931110 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:47.931385 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:47.931386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:47.931387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:47.931525 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:47.931587 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:47.931946 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 12:04:47.932046 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:04:47.932494 3576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:47.921012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41549 (local address 127.0.203.130:40585)
0619 12:04:47.921214 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:47.921270 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:47.922395 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:47.922818 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:47.922824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:47.923202 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:47.923528 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:47.923535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.924361 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.924362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:47.925638 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:47.925642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:47.926073 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:47.926075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:47.926255 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:47.926878 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:47.926888 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:47.928922 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:47.930632 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:47.930644 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:47.930647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:47.930917 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:47.931169 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:47.931170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:47.931170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:47.931296 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:47.931585 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:47.931592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:47.931802 (+ 210us) client_negotiation.cc:769] Sending connection context
0619 12:04:47.931916 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:04:47.932171 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
I20260619 12:04:47.933122 3574 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41549
I20260619 12:04:47.933283 3574 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:47.933533 3574 heartbeater.cc:511] Master 127.0.203.190:41549 requested a full tablet report, sending...
I20260619 12:04:47.934023 3249 ts_manager.cc:194] Registered new tserver with Master: 84d11212d86945ac968fca3e9e99dfa4 (127.0.203.130:36621)
I20260619 12:04:47.934454 3249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:40585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:47.946923 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41549
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:48.032668 3579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:48.032908 3579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:48.032933 3579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:48.034204 3579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:48.034256 3579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:48.034284 3579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:48.035969 3579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:48.036222 3579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:48.036446 3579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:48.041718 3587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:48.041743 3584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:48.041905 3585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:48.041997 3579 server_base.cc:1034] running on GCE node
I20260619 12:04:48.042300 3579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:48.042582 3579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:48.043769 3579 hybrid_clock.cc:648] HybridClock initialized: now 1781870688043740 us; error 50 us; skew 500 ppm
Jun 19 12:04:48 dist-test-slave-44vj krb5kdc[3204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870688, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:48.046577 3579 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:48.047094 3579 webserver.cc:460] Webserver started at http://127.0.203.131:37173/ using document root <none> and password file <none>
I20260619 12:04:48.047333 3579 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:48.047425 3579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:48.047567 3579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:48.048645 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "b19cece2bcfc4b1c8728467bd39da98c"
format_stamp: "Formatted at 2026-06-19 12:04:48 on dist-test-slave-44vj"
server_key: "c26ba9664a4fbd387f0eec82b3237ef0"
server_key_iv: "0c6ffff76dd29c04f9527c39d1b413a4"
server_key_version: "encryptionkey@0"
I20260619 12:04:48.048972 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "b19cece2bcfc4b1c8728467bd39da98c"
format_stamp: "Formatted at 2026-06-19 12:04:48 on dist-test-slave-44vj"
server_key: "c26ba9664a4fbd387f0eec82b3237ef0"
server_key_iv: "0c6ffff76dd29c04f9527c39d1b413a4"
server_key_version: "encryptionkey@0"
I20260619 12:04:48.050390 3579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:48.051275 3594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:48.051502 3579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:48.051607 3579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "b19cece2bcfc4b1c8728467bd39da98c"
format_stamp: "Formatted at 2026-06-19 12:04:48 on dist-test-slave-44vj"
server_key: "c26ba9664a4fbd387f0eec82b3237ef0"
server_key_iv: "0c6ffff76dd29c04f9527c39d1b413a4"
server_key_version: "encryptionkey@0"
I20260619 12:04:48.051689 3579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:48.083271 3579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:48.085598 3579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:48.085768 3579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:48.086023 3579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:48.086355 3579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:48.086391 3579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:48.086457 3579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:48.086481 3579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:48.093300 3579 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:44225
I20260619 12:04:48.093380 3707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:44225 every 8 connection(s)
I20260619 12:04:48.093720 3579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:04:48 dist-test-slave-44vj krb5kdc[3204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870688, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:48.103440 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 3579
I20260619 12:04:48.103555 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:48.103817 814 external_mini_cluster.cc:1428] Setting key e841834c606597125524c6a8990954da
I20260619 12:04:48.105031 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:48.094617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:45365 (local address 127.0.203.190:41549)
0619 12:04:48.094681 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:48.094683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:48.094860 (+ 177us) server_negotiation.cc:407] Connection header received
0619 12:04:48.096006 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:48.096007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:48.096020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:48.096109 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:48.096775 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:48.097350 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:48.098452 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:48.098655 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:48.098719 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:48.101443 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:48.101445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:48.101446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:48.101456 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:48.103046 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:48.103821 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:48.103822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:48.103822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:48.103918 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:48.104285 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:48.104286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:48.104287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:48.104436 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:48.104501 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:48.104819 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 12:04:48.104930 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:04:48.105195 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:48.094542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41549 (local address 127.0.203.131:45365)
0619 12:04:48.094708 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:48.094753 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:48.095861 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:48.096179 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:48.096183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:48.096451 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:48.096663 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:48.096667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:48.097485 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:48.097486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:48.098345 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:48.098348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:48.098767 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:48.098768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:48.098901 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:48.099557 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:48.099565 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:48.101284 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:48.103210 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:48.103245 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:48.103249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:48.103718 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:48.104015 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:48.104017 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:48.104018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:48.104203 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:48.104502 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:48.104510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:48.104674 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 12:04:48.104768 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 12:04:48.104960 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260619 12:04:48.105847 3708 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41549
I20260619 12:04:48.105978 3708 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:48.106187 3708 heartbeater.cc:511] Master 127.0.203.190:41549 requested a full tablet report, sending...
I20260619 12:04:48.106678 3249 ts_manager.cc:194] Registered new tserver with Master: b19cece2bcfc4b1c8728467bd39da98c (127.0.203.131:44225)
I20260619 12:04:48.107098 3249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:45365
I20260619 12:04:48.116016 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:04:48.129371 3288 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:48.128154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.0.203.190:41549)
0619 12:04:48.128265 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:48.128267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:48.128365 (+ 98us) server_negotiation.cc:407] Connection header received
0619 12:04:48.128426 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:48.128427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:48.128438 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:48.128500 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:48.129285 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57108: BlockingRecv error: recv got EOF from 127.0.0.1:57108 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 12:04:48.130406 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 3311
I20260619 12:04:48.134724 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 3445
I20260619 12:04:48.138856 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 3579
I20260619 12:04:48.143385 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 3217
2026-06-19T12:04:48Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1890 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:48 dist-test-slave-44vj krb5kdc[3720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:48 dist-test-slave-44vj krb5kdc[3720](info): set up 2 sockets
Jun 19 12:04:48 dist-test-slave-44vj krb5kdc[3720](info): commencing operation
krb5kdc: starting...
W20260619 12:04:49.103416 814 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:49 dist-test-slave-44vj krb5kdc[3720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:49.232168 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33569
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:49.324420 3733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:49.324755 3733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:49.324790 3733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:49.326460 3733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:49.326526 3733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:49.326540 3733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:49.326553 3733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:49.326565 3733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:49.328577 3733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:49.328917 3733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:49.329200 3733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:49.334923 3738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:49.334903 3739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:49.334947 3733 server_base.cc:1034] running on GCE node
W20260619 12:04:49.334903 3741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:49.335515 3733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:49.335882 3733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:49.337076 3733 hybrid_clock.cc:648] HybridClock initialized: now 1781870689337042 us; error 48 us; skew 500 ppm
Jun 19 12:04:49 dist-test-slave-44vj krb5kdc[3720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:49.344820 3733 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:49.345343 3733 webserver.cc:460] Webserver started at http://127.0.203.190:41063/ using document root <none> and password file <none>
I20260619 12:04:49.345570 3733 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:49.345620 3733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:49.345732 3733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:49.346854 3733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "fe08f4e4c91c46d186fa3b0feb79f01a"
format_stamp: "Formatted at 2026-06-19 12:04:49 on dist-test-slave-44vj"
server_key: "822149e1ab5c5c03879377c0d6a6da99"
server_key_iv: "5e2a7b011c6c8bb025aac425410ddc4f"
server_key_version: "encryptionkey@0"
I20260619 12:04:49.347172 3733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "fe08f4e4c91c46d186fa3b0feb79f01a"
format_stamp: "Formatted at 2026-06-19 12:04:49 on dist-test-slave-44vj"
server_key: "822149e1ab5c5c03879377c0d6a6da99"
server_key_iv: "5e2a7b011c6c8bb025aac425410ddc4f"
server_key_version: "encryptionkey@0"
I20260619 12:04:49.348688 3733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:04:49.349519 3748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:49.349815 3733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:49.349918 3733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "fe08f4e4c91c46d186fa3b0feb79f01a"
format_stamp: "Formatted at 2026-06-19 12:04:49 on dist-test-slave-44vj"
server_key: "822149e1ab5c5c03879377c0d6a6da99"
server_key_iv: "5e2a7b011c6c8bb025aac425410ddc4f"
server_key_version: "encryptionkey@0"
I20260619 12:04:49.349992 3733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:49.371004 3733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:49.373936 3733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:49.374104 3733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:49.378539 3733 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:33569
I20260619 12:04:49.378613 3800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:33569 every 8 connection(s)
I20260619 12:04:49.378969 3733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:49.380044 3801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:49.382905 3801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a: Bootstrap starting.
I20260619 12:04:49.383692 3801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:49.384018 3801 log.cc:824] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:49.384976 3801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a: No bootstrap required, opened a new log
I20260619 12:04:49.386804 3801 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33569 } }
I20260619 12:04:49.386976 3801 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:49.386998 3801 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe08f4e4c91c46d186fa3b0feb79f01a, State: Initialized, Role: FOLLOWER
I20260619 12:04:49.387148 3801 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33569 } }
I20260619 12:04:49.387233 3801 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:49.387270 3801 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:49.387303 3801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:49.388223 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 3733
I20260619 12:04:49.388278 3801 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33569 } }
I20260619 12:04:49.388350 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:49.388365 3801 leader_election.cc:302] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe08f4e4c91c46d186fa3b0feb79f01a; no voters:
I20260619 12:04:49.388545 814 external_mini_cluster.cc:1428] Setting key a80b63cb81767629adb95deafc8cf0b3
I20260619 12:04:49.388583 3801 leader_election.cc:288] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:49.388635 3804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:49.388794 3804 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [term 1 LEADER]: Becoming Leader. State: Replica: fe08f4e4c91c46d186fa3b0feb79f01a, State: Running, Role: LEADER
I20260619 12:04:49.388892 3804 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33569 } }
I20260619 12:04:49.388940 3801 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:49.389314 3806 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33569 } } }
I20260619 12:04:49.389688 3806 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:49.389611 3805 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe08f4e4c91c46d186fa3b0feb79f01a. Latest consensus state: current_term: 1 leader_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe08f4e4c91c46d186fa3b0feb79f01a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33569 } } }
I20260619 12:04:49.390007 3805 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:49.390232 3815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:49.390936 3815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:49.393703 3815 catalog_manager.cc:1347] Generated new cluster ID: 538d3f9b909c491a9d6e1e8a128053ce
I20260619 12:04:49.393771 3815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:04:49 dist-test-slave-44vj krb5kdc[3720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:49.403831 3815 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:49.404392 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:49.389271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.0.203.190:33569)
0619 12:04:49.389540 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:49.389552 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:49.389939 (+ 387us) server_negotiation.cc:407] Connection header received
0619 12:04:49.390200 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:49.390209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:49.390280 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:49.390410 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:49.391506 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:49.393670 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:49.395015 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:49.395814 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:49.395940 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:49.399486 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:49.399493 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:49.399493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:49.399512 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:49.401903 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:49.402358 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:49.402362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:49.402363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:49.402480 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:49.402680 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:49.402682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:49.402682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:49.403087 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:49.403259 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:49.403895 (+ 636us) server_negotiation.cc:299] Negotiation successful
0619 12:04:49.404126 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":129,"threads_started":1}
I20260619 12:04:49.404644 3815 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:49.405690 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:04:49.413100 3815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe08f4e4c91c46d186fa3b0feb79f01a: Generated new TSK 0
I20260619 12:04:49.413318 3815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:04:49.423604 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:49.418746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.0.203.190:33569)
0619 12:04:49.418831 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:49.418834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:49.418996 (+ 162us) server_negotiation.cc:407] Connection header received
0619 12:04:49.419106 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:49.419107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:49.419122 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:49.419176 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:49.419733 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:49.420263 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:49.421484 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:49.421867 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:49.421951 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:49.422863 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:49.422866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:49.422866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:49.423178 (+ 312us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:49.423204 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:49.423213 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:49.423282 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:49.423459 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 12:04:49.423497 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:04:49.427099 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 3733
2026-06-19T12:04:49Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1283 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:49 dist-test-slave-44vj krb5kdc[3834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:49 dist-test-slave-44vj krb5kdc[3834](info): set up 2 sockets
Jun 19 12:04:49 dist-test-slave-44vj krb5kdc[3834](info): commencing operation
krb5kdc: starting...
W20260619 12:04:50.391526 814 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:50 dist-test-slave-44vj krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:50.517777 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41985
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:41985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:50.603304 3847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:50.603572 3847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:50.603606 3847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:50.604890 3847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:50.604945 3847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:50.604964 3847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:50.604987 3847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:50.605005 3847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:50.606683 3847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:41985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:50.606961 3847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:50.607194 3847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:04:50.612617 3847 server_base.cc:1034] running on GCE node
W20260619 12:04:50.613647 3852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:50.613801 3853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:50.613883 3855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:50.614279 3847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:50.614698 3847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:50.615865 3847 hybrid_clock.cc:648] HybridClock initialized: now 1781870690615826 us; error 67 us; skew 500 ppm
Jun 19 12:04:50 dist-test-slave-44vj krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870690, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:50.618904 3847 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:50.619513 3847 webserver.cc:460] Webserver started at http://127.0.203.190:43651/ using document root <none> and password file <none>
I20260619 12:04:50.619752 3847 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:50.619803 3847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:50.619935 3847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:50.621163 3847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "b24fb3521b474fc2bfc401038276e0d1"
format_stamp: "Formatted at 2026-06-19 12:04:50 on dist-test-slave-44vj"
server_key: "20d31ed1deb6a67934f2421a2e8b7dda"
server_key_iv: "f8d91351cfdc856e7267932a80a6af71"
server_key_version: "encryptionkey@0"
I20260619 12:04:50.621544 3847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "b24fb3521b474fc2bfc401038276e0d1"
format_stamp: "Formatted at 2026-06-19 12:04:50 on dist-test-slave-44vj"
server_key: "20d31ed1deb6a67934f2421a2e8b7dda"
server_key_iv: "f8d91351cfdc856e7267932a80a6af71"
server_key_version: "encryptionkey@0"
I20260619 12:04:50.623090 3847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:50.624049 3862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:50.624418 3847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:04:50.624529 3847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "b24fb3521b474fc2bfc401038276e0d1"
format_stamp: "Formatted at 2026-06-19 12:04:50 on dist-test-slave-44vj"
server_key: "20d31ed1deb6a67934f2421a2e8b7dda"
server_key_iv: "f8d91351cfdc856e7267932a80a6af71"
server_key_version: "encryptionkey@0"
I20260619 12:04:50.624641 3847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:50.648089 3847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:50.650501 3847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:50.650632 3847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:50.654870 3847 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:41985
I20260619 12:04:50.654932 3914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:41985 every 8 connection(s)
I20260619 12:04:50.655288 3847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:50.656554 3915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:50.659278 3915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1: Bootstrap starting.
I20260619 12:04:50.659983 3915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:50.660259 3915 log.cc:824] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:50.661123 3915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1: No bootstrap required, opened a new log
I20260619 12:04:50.663089 3915 raft_consensus.cc:348] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24fb3521b474fc2bfc401038276e0d1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41985 } }
I20260619 12:04:50.663260 3915 raft_consensus.cc:374] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:50.663285 3915 raft_consensus.cc:729] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24fb3521b474fc2bfc401038276e0d1, State: Initialized, Role: FOLLOWER
I20260619 12:04:50.663478 3915 consensus_queue.cc:260] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24fb3521b474fc2bfc401038276e0d1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41985 } }
I20260619 12:04:50.663570 3915 raft_consensus.cc:388] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:50.663614 3915 raft_consensus.cc:482] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:50.663641 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 3847
I20260619 12:04:50.663727 3915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:50.663756 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:50.663945 814 external_mini_cluster.cc:1428] Setting key 0af934fbf49c8c531ed8683004a157f0
I20260619 12:04:50.664891 3915 raft_consensus.cc:504] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24fb3521b474fc2bfc401038276e0d1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41985 } }
I20260619 12:04:50.665006 3915 leader_election.cc:302] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b24fb3521b474fc2bfc401038276e0d1; no voters:
I20260619 12:04:50.665205 3915 leader_election.cc:288] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:50.665267 3920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:50.665475 3920 raft_consensus.cc:686] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [term 1 LEADER]: Becoming Leader. State: Replica: b24fb3521b474fc2bfc401038276e0d1, State: Running, Role: LEADER
I20260619 12:04:50.665582 3915 sys_catalog.cc:564] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:50.665655 3920 consensus_queue.cc:237] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24fb3521b474fc2bfc401038276e0d1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41985 } }
I20260619 12:04:50.666065 3920 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24fb3521b474fc2bfc401038276e0d1. Latest consensus state: current_term: 1 leader_uuid: "b24fb3521b474fc2bfc401038276e0d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24fb3521b474fc2bfc401038276e0d1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41985 } } }
I20260619 12:04:50.666148 3920 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:50.667253 3922 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b24fb3521b474fc2bfc401038276e0d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24fb3521b474fc2bfc401038276e0d1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41985 } } }
I20260619 12:04:50.667342 3922 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1 [sys.catalog]: This master's current role is: LEADER
W20260619 12:04:50.667773 3936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:04:50.667829 3936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:04:50.667874 3927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:50.668021 3927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:50.670043 3927 catalog_manager.cc:1347] Generated new cluster ID: df8dd96fd7fd45bd99eca9cd7663d242
I20260619 12:04:50.670113 3927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:04:50 dist-test-slave-44vj krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:50.676692 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:50.664567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45106 (local address 127.0.203.190:41985)
0619 12:04:50.664790 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:50.664799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:50.664836 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:04:50.664967 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:50.664974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:50.665031 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:50.665212 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:50.666027 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:50.667587 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:50.668502 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:50.669104 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:50.669226 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:50.672162 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:50.672168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:50.672168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:50.672184 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:50.674426 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:50.674881 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:50.674885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:50.674885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:50.675008 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:50.675174 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:50.675175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:50.675175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:50.675526 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:50.675721 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:50.676248 (+ 527us) server_negotiation.cc:299] Negotiation successful
0619 12:04:50.676416 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260619 12:04:50.687880 3927 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:50.688781 3927 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:50.697153 3927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b24fb3521b474fc2bfc401038276e0d1: Generated new TSK 0
I20260619 12:04:50.697455 3927 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:50.734365 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41985
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:50.844949 3941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:50.845198 3941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:50.845223 3941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:50.846457 3941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:50.846501 3941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:50.846529 3941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:50.848075 3941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:50.848296 3941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:50.848506 3941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:50.854224 3947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:50.854333 3946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:50.854404 3941 server_base.cc:1034] running on GCE node
W20260619 12:04:50.854256 3949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:50.854722 3941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:50.855007 3941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:50.856208 3941 hybrid_clock.cc:648] HybridClock initialized: now 1781870690856161 us; error 45 us; skew 500 ppm
Jun 19 12:04:50 dist-test-slave-44vj krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870690, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:50.859150 3941 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:50.859791 3941 webserver.cc:460] Webserver started at http://127.0.203.129:32835/ using document root <none> and password file <none>
I20260619 12:04:50.860023 3941 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:50.860074 3941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:50.860224 3941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:50.861445 3941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "831f9d6c26d3418e85be354cf1bbe27a"
format_stamp: "Formatted at 2026-06-19 12:04:50 on dist-test-slave-44vj"
server_key: "ab804df4d1c5ef664b4241af18818a88"
server_key_iv: "c391625ed2e7e7ac8d7eec8850bd4d84"
server_key_version: "encryptionkey@0"
I20260619 12:04:50.861835 3941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "831f9d6c26d3418e85be354cf1bbe27a"
format_stamp: "Formatted at 2026-06-19 12:04:50 on dist-test-slave-44vj"
server_key: "ab804df4d1c5ef664b4241af18818a88"
server_key_iv: "c391625ed2e7e7ac8d7eec8850bd4d84"
server_key_version: "encryptionkey@0"
I20260619 12:04:50.863504 3941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:50.864377 3956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:50.864637 3941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:50.864740 3941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "831f9d6c26d3418e85be354cf1bbe27a"
format_stamp: "Formatted at 2026-06-19 12:04:50 on dist-test-slave-44vj"
server_key: "ab804df4d1c5ef664b4241af18818a88"
server_key_iv: "c391625ed2e7e7ac8d7eec8850bd4d84"
server_key_version: "encryptionkey@0"
I20260619 12:04:50.864826 3941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:50.898552 3941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:50.901079 3941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:50.901257 3941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:50.901574 3941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:50.901934 3941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:50.901971 3941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:50.901994 3941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:50.902006 3941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:50.909058 3941 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:37121
I20260619 12:04:50.909142 4069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:37121 every 8 connection(s)
I20260619 12:04:50.909498 3941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:04:50.911556 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 3941
I20260619 12:04:50.911662 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:50.911850 814 external_mini_cluster.cc:1428] Setting key 81aa67defbefc54c61686b8532aba0a2
Jun 19 12:04:50 dist-test-slave-44vj krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870690, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
I20260619 12:04:50.921152 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:50.910417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:51439 (local address 127.0.203.190:41985)
0619 12:04:50.910498 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:50.910500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:50.910714 (+ 214us) server_negotiation.cc:407] Connection header received
0619 12:04:50.911285 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:50.911287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:50.911308 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:50.911480 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:50.912631 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:50.913186 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:50.914228 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:50.914442 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:50.914499 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:50.917543 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:50.917545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:50.917546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:50.917556 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:50.919235 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:50.919846 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:50.919848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:50.919848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:50.919928 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:50.920195 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:50.920196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:50.920197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:50.920346 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:50.920432 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:50.920925 (+ 493us) server_negotiation.cc:299] Negotiation successful
0619 12:04:50.921045 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:04:50.921301 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:50.910361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41985 (local address 127.0.203.129:51439)
0619 12:04:50.910542 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:50.910589 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:50.911161 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:50.911549 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:50.911553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:50.912273 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:50.912537 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:50.912544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:50.913275 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:50.913276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:50.914137 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:50.914139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:50.914509 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:50.914510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:50.914658 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:50.915237 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:50.915248 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:50.917401 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:50.919413 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:50.919428 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:50.919431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:50.919754 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:50.919997 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:50.919998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:50.919998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:50.920129 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:50.920457 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:50.920471 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:50.920717 (+ 246us) client_negotiation.cc:769] Sending connection context
0619 12:04:50.920847 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 12:04:50.921023 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260619 12:04:50.922196 4070 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41985
I20260619 12:04:50.922322 4070 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:50.922552 4070 heartbeater.cc:511] Master 127.0.203.190:41985 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
I20260619 12:04:50.923436 3879 ts_manager.cc:194] Registered new tserver with Master: 831f9d6c26d3418e85be354cf1bbe27a (127.0.203.129:37121)
I20260619 12:04:50.924541 3879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:51439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:50.940999 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41985
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:51.032343 4075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:51.032599 4075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:51.032625 4075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:51.033949 4075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:51.034001 4075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:51.034031 4075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:51.035670 4075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:51.036013 4075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:51.036300 4075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:51.041993 4083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:51.042008 4081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:51.042008 4080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:51.042582 4075 server_base.cc:1034] running on GCE node
I20260619 12:04:51.042855 4075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:51.043123 4075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:51.044314 4075 hybrid_clock.cc:648] HybridClock initialized: now 1781870691044225 us; error 107 us; skew 500 ppm
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:51.047457 4075 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:51.048177 4075 webserver.cc:460] Webserver started at http://127.0.203.130:45563/ using document root <none> and password file <none>
I20260619 12:04:51.048431 4075 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:51.048482 4075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:51.048617 4075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:51.049827 4075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "58bc7a832d42487e8a8e64e1a0ffc819"
format_stamp: "Formatted at 2026-06-19 12:04:51 on dist-test-slave-44vj"
server_key: "211b0385e2c296f997c2c74af3d9284b"
server_key_iv: "a41ff53d895f8fa6d370e47b84a11aa8"
server_key_version: "encryptionkey@0"
I20260619 12:04:51.050222 4075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "58bc7a832d42487e8a8e64e1a0ffc819"
format_stamp: "Formatted at 2026-06-19 12:04:51 on dist-test-slave-44vj"
server_key: "211b0385e2c296f997c2c74af3d9284b"
server_key_iv: "a41ff53d895f8fa6d370e47b84a11aa8"
server_key_version: "encryptionkey@0"
I20260619 12:04:51.051849 4075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:51.052728 4090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:51.052964 4075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:04:51.053053 4075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "58bc7a832d42487e8a8e64e1a0ffc819"
format_stamp: "Formatted at 2026-06-19 12:04:51 on dist-test-slave-44vj"
server_key: "211b0385e2c296f997c2c74af3d9284b"
server_key_iv: "a41ff53d895f8fa6d370e47b84a11aa8"
server_key_version: "encryptionkey@0"
I20260619 12:04:51.053133 4075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:51.060292 4075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:51.062885 4075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:51.063038 4075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:51.063349 4075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:51.063777 4075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:51.063815 4075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:51.063839 4075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:51.063851 4075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:51.071566 4075 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:45813
I20260619 12:04:51.071662 4203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:45813 every 8 connection(s)
I20260619 12:04:51.072005 4075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:51.077824 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4075
I20260619 12:04:51.077981 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:51.078186 814 external_mini_cluster.cc:1428] Setting key 0b3129afc8e8bcd3bde8ed60d9f30261
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:51.082742 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:51.072970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:42749 (local address 127.0.203.190:41985)
0619 12:04:51.073069 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:51.073072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:51.073286 (+ 214us) server_negotiation.cc:407] Connection header received
0619 12:04:51.073889 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:51.073890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:51.073903 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:51.073958 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:51.075055 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.075620 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.076674 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.076866 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.076924 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:51.079758 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:51.079760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:51.079760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:51.079771 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:51.081198 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:51.081677 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:51.081679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:51.081679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:51.081754 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:51.081988 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:51.081989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:51.081989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:51.082123 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:51.082192 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:51.082520 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 12:04:51.082626 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:04:51.082927 4206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:51.072917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41985 (local address 127.0.203.130:42749)
0619 12:04:51.073133 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:51.073181 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:51.073778 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:51.074039 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:51.074043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:51.074702 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:51.074935 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:51.074940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.075723 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.075724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:51.076585 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:51.076588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.076940 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.076941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:51.077066 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:51.077697 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:51.077708 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:51.079593 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:51.081297 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:51.081306 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:51.081309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:51.081593 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:51.081825 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:51.081826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:51.081826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:51.081930 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:51.082200 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:51.082206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:51.082368 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 12:04:51.082470 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 12:04:51.082647 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260619 12:04:51.083642 4204 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41985
I20260619 12:04:51.083794 4204 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:51.084057 4204 heartbeater.cc:511] Master 127.0.203.190:41985 requested a full tablet report, sending...
I20260619 12:04:51.084542 3879 ts_manager.cc:194] Registered new tserver with Master: 58bc7a832d42487e8a8e64e1a0ffc819 (127.0.203.130:45813)
I20260619 12:04:51.085037 3879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:42749
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:51.104569 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41985
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:51.194288 4209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:51.194523 4209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:51.194558 4209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:51.195855 4209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:51.195910 4209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:51.195956 4209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:51.197665 4209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:51.197944 4209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:51.198155 4209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:51.203771 4217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:51.203781 4214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:51.203783 4215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:51.204249 4209 server_base.cc:1034] running on GCE node
I20260619 12:04:51.204588 4209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:51.204892 4209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:51.206096 4209 hybrid_clock.cc:648] HybridClock initialized: now 1781870691206069 us; error 40 us; skew 500 ppm
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:51.209172 4209 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:51.209697 4209 webserver.cc:460] Webserver started at http://127.0.203.131:43499/ using document root <none> and password file <none>
I20260619 12:04:51.209911 4209 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:51.209959 4209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:51.210068 4209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:51.211136 4209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "fecb84749f37497899b3a24d520c6506"
format_stamp: "Formatted at 2026-06-19 12:04:51 on dist-test-slave-44vj"
server_key: "3cfde5527f02295ffa82c72dc9e3212b"
server_key_iv: "bb96929d89fd34b14a7613e7cf0c2130"
server_key_version: "encryptionkey@0"
I20260619 12:04:51.211503 4209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "fecb84749f37497899b3a24d520c6506"
format_stamp: "Formatted at 2026-06-19 12:04:51 on dist-test-slave-44vj"
server_key: "3cfde5527f02295ffa82c72dc9e3212b"
server_key_iv: "bb96929d89fd34b14a7613e7cf0c2130"
server_key_version: "encryptionkey@0"
I20260619 12:04:51.213227 4209 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:04:51.214213 4224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:51.214553 4209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:51.214670 4209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "fecb84749f37497899b3a24d520c6506"
format_stamp: "Formatted at 2026-06-19 12:04:51 on dist-test-slave-44vj"
server_key: "3cfde5527f02295ffa82c72dc9e3212b"
server_key_iv: "bb96929d89fd34b14a7613e7cf0c2130"
server_key_version: "encryptionkey@0"
I20260619 12:04:51.214780 4209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:51.235826 4209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:51.238209 4209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:51.238344 4209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:51.238629 4209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:51.238984 4209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:51.239019 4209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:51.239045 4209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:51.239069 4209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:51.246133 4209 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:35171
I20260619 12:04:51.246232 4337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:35171 every 8 connection(s)
I20260619 12:04:51.246553 4209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:04:51.250629 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4209
I20260619 12:04:51.250723 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:51.250919 814 external_mini_cluster.cc:1428] Setting key 16d7cf7855280375d0a8ed07e3c90b01
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:51.257256 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:51.247418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:53717 (local address 127.0.203.190:41985)
0619 12:04:51.247503 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:51.247507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:51.247755 (+ 248us) server_negotiation.cc:407] Connection header received
0619 12:04:51.248387 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:51.248388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:51.248399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:51.248503 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:51.249578 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.250094 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.251433 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.251672 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.251748 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:51.254266 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:51.254268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:51.254269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:51.254280 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:51.255582 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:51.256067 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:51.256068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:51.256068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:51.256142 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:51.256552 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:51.256553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:51.256554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:51.256678 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:51.256738 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:51.257056 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 12:04:51.257159 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:04:51.257484 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:51.247352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41985 (local address 127.0.203.131:53717)
0619 12:04:51.247605 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:51.247647 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:51.248258 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:51.248571 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:51.248575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:51.249188 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:51.249456 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:51.249461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.250178 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.250178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:51.251321 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:51.251324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.251772 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.251773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:51.251948 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:51.252540 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:51.252549 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:51.254122 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:51.255677 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:51.255683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:51.255685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:51.255970 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:51.256275 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:51.256277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:51.256277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:51.256477 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:51.256742 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:51.256750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:51.256922 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 12:04:51.257025 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 12:04:51.257234 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":133,"threads_started":1}
I20260619 12:04:51.258069 4338 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41985
I20260619 12:04:51.258170 4338 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:51.258345 4338 heartbeater.cc:511] Master 127.0.203.190:41985 requested a full tablet report, sending...
I20260619 12:04:51.258733 3879 ts_manager.cc:194] Registered new tserver with Master: fecb84749f37497899b3a24d520c6506 (127.0.203.131:35171)
I20260619 12:04:51.259166 3879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:53717
I20260619 12:04:51.263583 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870691, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870691, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:51.287756 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:51.280045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.0.203.190:41985)
0619 12:04:51.280149 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:51.280152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:51.280233 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:04:51.280302 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:51.280303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:51.280316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:51.280376 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:51.280922 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.281470 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.282251 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:51.282444 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:51.282495 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:51.284943 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:51.284945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:51.284945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:51.284954 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:51.286257 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:51.286762 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:51.286763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:51.286763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:51.286811 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:51.287079 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:51.287080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:51.287080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:51.287246 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:51.287332 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:51.287607 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 12:04:51.287677 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 12:04:51.288066 3879 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45114
I20260619 12:04:51.288556 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 3941
I20260619 12:04:51.292758 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4075
I20260619 12:04:51.297221 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4209
I20260619 12:04:51.302306 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 3847
2026-06-19T12:04:51Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1875 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[4350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[4350](info): set up 2 sockets
Jun 19 12:04:51 dist-test-slave-44vj krb5kdc[4350](info): commencing operation
krb5kdc: starting...
W20260619 12:04:52.267616 814 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:52 dist-test-slave-44vj krb5kdc[4350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:52.390367 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34007
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:34007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:52.481513 4363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:52.481806 4363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:52.481842 4363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:52.483175 4363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:52.483223 4363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:52.483242 4363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:52.483259 4363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:52.483274 4363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:52.484930 4363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:34007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:52.485213 4363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:52.485491 4363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:52.491415 4371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:52.491688 4363 server_base.cc:1034] running on GCE node
W20260619 12:04:52.491428 4368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:52.491418 4369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:52.492362 4363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:52.492763 4363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:52.493919 4363 hybrid_clock.cc:648] HybridClock initialized: now 1781870692493886 us; error 62 us; skew 500 ppm
Jun 19 12:04:52 dist-test-slave-44vj krb5kdc[4350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:52.497104 4363 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:52.497732 4363 webserver.cc:460] Webserver started at http://127.0.203.190:34089/ using document root <none> and password file <none>
I20260619 12:04:52.497989 4363 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:52.498039 4363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:52.498175 4363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:52.499444 4363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "f4074f832bdd4380ab8e57b2b31b3bcb"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "95536f7636482a4417db22808fce883e"
server_key_iv: "3163f4706714a79157c9a3c4b234e8c6"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.499820 4363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "f4074f832bdd4380ab8e57b2b31b3bcb"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "95536f7636482a4417db22808fce883e"
server_key_iv: "3163f4706714a79157c9a3c4b234e8c6"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.501410 4363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:04:52.502401 4378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:52.502663 4363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:04:52.502763 4363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "f4074f832bdd4380ab8e57b2b31b3bcb"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "95536f7636482a4417db22808fce883e"
server_key_iv: "3163f4706714a79157c9a3c4b234e8c6"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.502832 4363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:52.526131 4363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:52.529261 4363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:52.529451 4363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:52.534457 4363 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:34007
I20260619 12:04:52.534528 4430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:34007 every 8 connection(s)
I20260619 12:04:52.535086 4363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:52.535969 4431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:52.536356 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4363
I20260619 12:04:52.536460 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:52.536659 814 external_mini_cluster.cc:1428] Setting key bf79455c1c62006e3df108aaa5e4a214
I20260619 12:04:52.539448 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb: Bootstrap starting.
I20260619 12:04:52.540150 4431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:52.540455 4431 log.cc:824] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:52.541205 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb: No bootstrap required, opened a new log
I20260619 12:04:52.543419 4431 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34007 } }
I20260619 12:04:52.543604 4431 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:52.543640 4431 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4074f832bdd4380ab8e57b2b31b3bcb, State: Initialized, Role: FOLLOWER
I20260619 12:04:52.543785 4431 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34007 } }
I20260619 12:04:52.543862 4431 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:52.543896 4431 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:52.543944 4431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 0 FOLLOWER]: Advancing to term 1
Jun 19 12:04:52 dist-test-slave-44vj krb5kdc[4350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:52.544633 4431 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34007 } }
I20260619 12:04:52.544747 4431 leader_election.cc:302] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4074f832bdd4380ab8e57b2b31b3bcb; no voters:
I20260619 12:04:52.544937 4431 leader_election.cc:288] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:52.545025 4436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:52.545255 4431 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:52.545257 4436 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [term 1 LEADER]: Becoming Leader. State: Replica: f4074f832bdd4380ab8e57b2b31b3bcb, State: Running, Role: LEADER
I20260619 12:04:52.545429 4436 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34007 } }
I20260619 12:04:52.545933 4438 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4074f832bdd4380ab8e57b2b31b3bcb. Latest consensus state: current_term: 1 leader_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34007 } } }
I20260619 12:04:52.546026 4438 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:52.546265 4445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:52.546494 4437 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4074f832bdd4380ab8e57b2b31b3bcb" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34007 } } }
I20260619 12:04:52.546576 4437 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:52.546950 4445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:52.548951 4445 catalog_manager.cc:1347] Generated new cluster ID: 3be87ccdd19144c2bcc040b53716cb90
I20260619 12:04:52.549021 4445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:52.554845 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:52.537285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.0.203.190:34007)
0619 12:04:52.537561 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:52.537571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:52.537694 (+ 123us) server_negotiation.cc:407] Connection header received
0619 12:04:52.538288 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:52.538297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:52.538354 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:52.538493 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:52.539351 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.540515 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.541331 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.541792 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.541897 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:52.544693 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:52.544698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:52.544699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:52.544712 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:52.552780 (+ 8068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:52.553447 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:52.553451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:52.553451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:52.553575 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:52.553802 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:52.553803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:52.553803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:52.554142 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:52.554325 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:52.554447 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 12:04:52.554605 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":126,"threads_started":1}
I20260619 12:04:52.556726 4445 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:52.557536 4445 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:52.562250 4445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4074f832bdd4380ab8e57b2b31b3bcb: Generated new TSK 0
I20260619 12:04:52.562443 4445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:52.590101 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34007
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:52.681416 4457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:52.681690 4457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:52.681717 4457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:52.683008 4457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:52.683063 4457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:52.683091 4457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:52.684719 4457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:52.684989 4457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:52.685264 4457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:52.690820 4465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:52.690816 4463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:52.691061 4457 server_base.cc:1034] running on GCE node
W20260619 12:04:52.690827 4462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:52.691416 4457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:52.691686 4457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:52.692857 4457 hybrid_clock.cc:648] HybridClock initialized: now 1781870692692841 us; error 39 us; skew 500 ppm
Jun 19 12:04:52 dist-test-slave-44vj krb5kdc[4350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:52.695967 4457 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:52.696563 4457 webserver.cc:460] Webserver started at http://127.0.203.129:46471/ using document root <none> and password file <none>
I20260619 12:04:52.696800 4457 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:52.696945 4457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:52.697120 4457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:52.698366 4457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "7cec51da83f84c9693275c8bb6bff733"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "13c1709f4a76472ccf0e429ea17a2eac"
server_key_iv: "418e3ea48ccaf6a8ea0361c5e572f8d6"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.698733 4457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "7cec51da83f84c9693275c8bb6bff733"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "13c1709f4a76472ccf0e429ea17a2eac"
server_key_iv: "418e3ea48ccaf6a8ea0361c5e572f8d6"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.700420 4457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:52.701555 4472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:52.701800 4457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:52.701908 4457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "7cec51da83f84c9693275c8bb6bff733"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "13c1709f4a76472ccf0e429ea17a2eac"
server_key_iv: "418e3ea48ccaf6a8ea0361c5e572f8d6"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.702020 4457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:52.713158 4457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:52.715324 4457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:52.715528 4457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:52.715819 4457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:52.716166 4457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:52.716204 4457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:52.716228 4457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:52.716256 4457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:52.722707 4457 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:46297
I20260619 12:04:52.722810 4585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:46297 every 8 connection(s)
I20260619 12:04:52.723134 4457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:04:52.725828 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4457
I20260619 12:04:52.725925 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:52.726126 814 external_mini_cluster.cc:1428] Setting key 39eb5ab5605c6d06e52468b48b500486
Jun 19 12:04:52 dist-test-slave-44vj krb5kdc[4350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
I20260619 12:04:52.734671 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:52.724107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:47871 (local address 127.0.203.190:34007)
0619 12:04:52.724169 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:52.724171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:52.724424 (+ 253us) server_negotiation.cc:407] Connection header received
0619 12:04:52.725514 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:52.725516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:52.725531 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:52.725619 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:52.726474 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.727178 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.728588 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.728810 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.728876 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:52.731801 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:52.731802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:52.731803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:52.731812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:52.733156 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:52.733665 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:52.733666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:52.733667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:52.733727 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:52.733971 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:52.733972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:52.733972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:52.734100 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:52.734163 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:52.734461 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 12:04:52.734571 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:04:52.734877 4588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:52.724072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34007 (local address 127.0.203.129:47871)
0619 12:04:52.724243 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:52.724310 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:52.725396 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:52.725702 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:52.725707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:52.726063 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:52.726352 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:52.726360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.727326 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.727327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:52.728478 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:52.728482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.728907 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.728908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:52.729075 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:52.729715 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:52.729726 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:52.731689 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:52.733263 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:52.733278 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:52.733282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:52.733591 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:52.733785 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:52.733786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:52.733786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:52.733914 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:52.734158 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:52.734164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:52.734332 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 12:04:52.734433 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:04:52.734635 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260619 12:04:52.735487 4586 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34007
I20260619 12:04:52.735642 4586 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:52.735961 4586 heartbeater.cc:511] Master 127.0.203.190:34007 requested a full tablet report, sending...
I20260619 12:04:52.736711 4395 ts_manager.cc:194] Registered new tserver with Master: 7cec51da83f84c9693275c8bb6bff733 (127.0.203.129:46297)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
I20260619 12:04:52.737684 4395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:47871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:52.755270 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34007
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:52.845769 4591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:52.846045 4591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:52.846083 4591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:52.847429 4591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:52.847478 4591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:52.847517 4591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:52.849160 4591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:52.849452 4591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:52.849694 4591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:52.855113 4596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:52.855129 4599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:52.855139 4597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:52.855441 4591 server_base.cc:1034] running on GCE node
I20260619 12:04:52.855806 4591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:52.856076 4591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:52.857226 4591 hybrid_clock.cc:648] HybridClock initialized: now 1781870692857202 us; error 47 us; skew 500 ppm
Jun 19 12:04:52 dist-test-slave-44vj krb5kdc[4350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:52.860085 4591 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:52.860720 4591 webserver.cc:460] Webserver started at http://127.0.203.130:42507/ using document root <none> and password file <none>
I20260619 12:04:52.860965 4591 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:52.861011 4591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:52.861117 4591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:52.862260 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "36cfcd87668e44da921ba9a46a1980e3"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "bed396ab31c45cd54a3e2f257462bfc4"
server_key_iv: "b9ce92061a53cf2eb71dbde64bb112a7"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.862602 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "36cfcd87668e44da921ba9a46a1980e3"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "bed396ab31c45cd54a3e2f257462bfc4"
server_key_iv: "b9ce92061a53cf2eb71dbde64bb112a7"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.864056 4591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:52.864946 4606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:52.865177 4591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:52.865268 4591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "36cfcd87668e44da921ba9a46a1980e3"
format_stamp: "Formatted at 2026-06-19 12:04:52 on dist-test-slave-44vj"
server_key: "bed396ab31c45cd54a3e2f257462bfc4"
server_key_iv: "b9ce92061a53cf2eb71dbde64bb112a7"
server_key_version: "encryptionkey@0"
I20260619 12:04:52.865363 4591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:52.885861 4591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:52.888367 4591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:52.888510 4591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:52.888777 4591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:52.889113 4591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:52.889148 4591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:52.889180 4591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:52.889199 4591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:52.896230 4591 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:46019
I20260619 12:04:52.896306 4719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:46019 every 8 connection(s)
I20260619 12:04:52.896661 4591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:52.901463 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4591
I20260619 12:04:52.901566 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:52.901772 814 external_mini_cluster.cc:1428] Setting key 94f9bc811bee76ff6014050f5e4895ee
Jun 19 12:04:52 dist-test-slave-44vj krb5kdc[4350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:52.907605 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:52.897491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:45467 (local address 127.0.203.190:34007)
0619 12:04:52.897582 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:52.897585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:52.897783 (+ 198us) server_negotiation.cc:407] Connection header received
0619 12:04:52.898827 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:52.898829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:52.898841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:52.898916 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:52.899687 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.900214 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.901296 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.901501 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.901598 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:52.904206 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:52.904209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:52.904209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:52.904221 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:52.905910 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:52.906400 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:52.906401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:52.906402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:52.906501 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:52.906812 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:52.906812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:52.906813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:52.906969 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:52.907051 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:52.907392 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 12:04:52.907497 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:04:52.907701 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:52.897444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34007 (local address 127.0.203.130:45467)
0619 12:04:52.897632 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:52.897675 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:52.898713 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:52.898982 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:52.898986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:52.899270 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:52.899592 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:52.899598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.900324 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.900324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:52.901192 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:52.901193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:52.901615 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:52.901616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:52.901772 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:52.902374 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:52.902384 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:52.904061 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:52.906001 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:52.906007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:52.906009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:52.906325 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:52.906568 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:52.906569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:52.906569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:52.906745 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:52.907045 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:52.907049 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:52.907195 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 12:04:52.907317 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 12:04:52.907503 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":123,"threads_started":1}
I20260619 12:04:52.908488 4720 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34007
I20260619 12:04:52.908645 4720 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:52.908918 4720 heartbeater.cc:511] Master 127.0.203.190:34007 requested a full tablet report, sending...
I20260619 12:04:52.909379 4395 ts_manager.cc:194] Registered new tserver with Master: 36cfcd87668e44da921ba9a46a1980e3 (127.0.203.130:46019)
I20260619 12:04:52.909915 4395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:45467
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:52.930320 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34007
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:53.017951 4725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:53.018174 4725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:53.018199 4725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:53.019412 4725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:53.019466 4725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:53.019492 4725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:53.020992 4725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:53.021216 4725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:53.021415 4725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:53.026708 4733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:53.026674 4730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:53.026690 4731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:53.027099 4725 server_base.cc:1034] running on GCE node
I20260619 12:04:53.027385 4725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:53.027716 4725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:53.028894 4725 hybrid_clock.cc:648] HybridClock initialized: now 1781870693028869 us; error 44 us; skew 500 ppm
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870693, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:53.032538 4725 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:53.033068 4725 webserver.cc:460] Webserver started at http://127.0.203.131:46433/ using document root <none> and password file <none>
I20260619 12:04:53.033324 4725 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:53.033377 4725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:53.033495 4725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:53.034564 4725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "c01635cc62d9472ba96b07a5b846b39f"
format_stamp: "Formatted at 2026-06-19 12:04:53 on dist-test-slave-44vj"
server_key: "95d3963db4ad47e4ac6ed77fcbb57f6a"
server_key_iv: "f82eb5dcf23a90d10b549a999be0bd81"
server_key_version: "encryptionkey@0"
I20260619 12:04:53.034895 4725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "c01635cc62d9472ba96b07a5b846b39f"
format_stamp: "Formatted at 2026-06-19 12:04:53 on dist-test-slave-44vj"
server_key: "95d3963db4ad47e4ac6ed77fcbb57f6a"
server_key_iv: "f82eb5dcf23a90d10b549a999be0bd81"
server_key_version: "encryptionkey@0"
I20260619 12:04:53.036415 4725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:53.037222 4740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:53.037453 4725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:53.037549 4725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "c01635cc62d9472ba96b07a5b846b39f"
format_stamp: "Formatted at 2026-06-19 12:04:53 on dist-test-slave-44vj"
server_key: "95d3963db4ad47e4ac6ed77fcbb57f6a"
server_key_iv: "f82eb5dcf23a90d10b549a999be0bd81"
server_key_version: "encryptionkey@0"
I20260619 12:04:53.037626 4725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:53.059729 4725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:53.062790 4725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:53.062958 4725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:53.063253 4725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:53.063656 4725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:53.063704 4725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:53.063728 4725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:53.063742 4725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:53.071524 4725 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:40423
I20260619 12:04:53.071588 4853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:40423 every 8 connection(s)
I20260619 12:04:53.071969 4725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:04:53.076521 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4725
I20260619 12:04:53.076630 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:53.076835 814 external_mini_cluster.cc:1428] Setting key bff9bc179e876dce8644fd55e19f5540
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870693, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:53.082645 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:53.072865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:54955 (local address 127.0.203.190:34007)
0619 12:04:53.072930 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:53.072933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:53.073141 (+ 208us) server_negotiation.cc:407] Connection header received
0619 12:04:53.074135 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:53.074136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:53.074151 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:53.074253 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:53.075094 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.075748 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.077075 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.077320 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.077396 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:53.079860 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:53.079861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:53.079862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:53.079870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:53.081229 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:53.081705 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:53.081706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:53.081706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:53.081759 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:53.082037 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:53.082038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:53.082038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:53.082156 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:53.082208 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:53.082480 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 12:04:53.082558 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:04:53.082830 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:53.072794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34007 (local address 127.0.203.131:54955)
0619 12:04:53.072986 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:53.073033 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:53.074001 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:53.074385 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:53.074389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:53.074728 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:53.074947 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:53.074952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.075940 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.075942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:53.076963 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:53.076968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.077416 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.077416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:53.077591 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:53.078187 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:53.078197 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:53.079757 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:53.081339 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:53.081348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:53.081350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:53.081626 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:53.081854 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:53.081855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:53.081855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:53.081981 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:53.082205 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:53.082209 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:53.082351 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 12:04:53.082444 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 12:04:53.082591 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260619 12:04:53.083539 4854 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34007
I20260619 12:04:53.083645 4854 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:53.083839 4854 heartbeater.cc:511] Master 127.0.203.190:34007 requested a full tablet report, sending...
I20260619 12:04:53.084368 4395 ts_manager.cc:194] Registered new tserver with Master: c01635cc62d9472ba96b07a5b846b39f (127.0.203.131:40423)
I20260619 12:04:53.084815 4395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:54955
I20260619 12:04:53.088907 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:53.113399 4861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:53.105361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35396 (local address 127.0.203.129:46297)
0619 12:04:53.105581 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:53.105588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:53.105601 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:04:53.105631 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:53.105642 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:53.105657 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:53.105724 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:53.106418 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.107236 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.107992 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.108317 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.108379 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:53.110735 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:53.110744 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:53.110745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:53.110752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:53.112318 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:53.112630 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:53.112641 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:53.112642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:53.112693 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:53.112855 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:53.112856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:53.112856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:53.112991 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:53.113147 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:53.113216 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 12:04:53.113292 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:53.122000 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:53.114942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33432 (local address 127.0.203.190:34007)
0619 12:04:53.115012 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:53.115014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:53.115059 (+ 45us) server_negotiation.cc:407] Connection header received
0619 12:04:53.115126 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:53.115126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:53.115138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:53.115182 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:53.115761 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.116347 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.117041 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:53.117256 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:53.117324 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:53.119637 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:53.119638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:53.119639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:53.119647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:53.120970 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:53.121291 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:53.121292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:53.121293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:53.121346 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:53.121499 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:53.121499 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:53.121499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:53.121657 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:53.121748 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:53.121827 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 12:04:53.121898 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 12:04:53.122319 4395 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33432
I20260619 12:04:53.122773 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4457
I20260619 12:04:53.127934 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4591
I20260619 12:04:53.132630 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4725
I20260619 12:04:53.137023 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4363
2026-06-19T12:04:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1834 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4866](info): set up 2 sockets
Jun 19 12:04:53 dist-test-slave-44vj krb5kdc[4866](info): commencing operation
krb5kdc: starting...
W20260619 12:04:54.111671 814 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:54.238883 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38755
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:38755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:54.327541 4879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:54.327772 4879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:54.327798 4879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:54.329069 4879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:54.329118 4879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:54.329133 4879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:54.329144 4879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:54.329155 4879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:54.330761 4879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:38755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:54.330989 4879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:54.331197 4879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:54.336565 4884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:54.336575 4887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:54.336575 4885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:54.336963 4879 server_base.cc:1034] running on GCE node
I20260619 12:04:54.337261 4879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:54.337703 4879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:54.338896 4879 hybrid_clock.cc:648] HybridClock initialized: now 1781870694338865 us; error 54 us; skew 500 ppm
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:54.341838 4879 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:54.342272 4879 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:54.342332 4879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:54.342475 4879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:54.343777 4879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "e55731f17ac840118ae43499be6f40a9"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "c17089de674b0132e5fc1a29dc8ee963"
server_key_iv: "3a382282deebb0b14b2bff3d92c4184e"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.344132 4879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "e55731f17ac840118ae43499be6f40a9"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "c17089de674b0132e5fc1a29dc8ee963"
server_key_iv: "3a382282deebb0b14b2bff3d92c4184e"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.345542 4879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:04:54.346449 4893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.346740 4879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:54.346886 4879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "e55731f17ac840118ae43499be6f40a9"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "c17089de674b0132e5fc1a29dc8ee963"
server_key_iv: "3a382282deebb0b14b2bff3d92c4184e"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.346963 4879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:54.372028 4879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:54.374593 4879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:54.374744 4879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:54.379402 4879 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:38755
I20260619 12:04:54.379484 4945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:38755 every 8 connection(s)
I20260619 12:04:54.379768 4879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:54.380815 4946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:54.383311 4946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9: Bootstrap starting.
I20260619 12:04:54.383960 4946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:54.384217 4946 log.cc:824] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:54.385022 4946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9: No bootstrap required, opened a new log
I20260619 12:04:54.385149 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4879
I20260619 12:04:54.385252 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:54.385429 814 external_mini_cluster.cc:1428] Setting key eb5aa3f44d612b18cfd63003f6a4c349
I20260619 12:04:54.387024 4946 raft_consensus.cc:348] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55731f17ac840118ae43499be6f40a9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38755 } }
I20260619 12:04:54.387181 4946 raft_consensus.cc:374] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:54.387204 4946 raft_consensus.cc:729] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e55731f17ac840118ae43499be6f40a9, State: Initialized, Role: FOLLOWER
I20260619 12:04:54.387382 4946 consensus_queue.cc:260] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55731f17ac840118ae43499be6f40a9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38755 } }
I20260619 12:04:54.387475 4946 raft_consensus.cc:388] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:54.387514 4946 raft_consensus.cc:482] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:54.387557 4946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:54.388320 4946 raft_consensus.cc:504] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55731f17ac840118ae43499be6f40a9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38755 } }
I20260619 12:04:54.388422 4946 leader_election.cc:302] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e55731f17ac840118ae43499be6f40a9; no voters:
I20260619 12:04:54.388598 4946 leader_election.cc:288] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:54.388688 4951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:54.388890 4946 sys_catalog.cc:564] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:54.388903 4951 raft_consensus.cc:686] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [term 1 LEADER]: Becoming Leader. State: Replica: e55731f17ac840118ae43499be6f40a9, State: Running, Role: LEADER
I20260619 12:04:54.389068 4951 consensus_queue.cc:237] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55731f17ac840118ae43499be6f40a9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38755 } }
I20260619 12:04:54.389582 4953 sys_catalog.cc:455] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e55731f17ac840118ae43499be6f40a9. Latest consensus state: current_term: 1 leader_uuid: "e55731f17ac840118ae43499be6f40a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55731f17ac840118ae43499be6f40a9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38755 } } }
I20260619 12:04:54.389684 4953 sys_catalog.cc:458] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:54.390015 4952 sys_catalog.cc:455] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e55731f17ac840118ae43499be6f40a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55731f17ac840118ae43499be6f40a9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38755 } } }
I20260619 12:04:54.390115 4952 sys_catalog.cc:458] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9 [sys.catalog]: This master's current role is: LEADER
W20260619 12:04:54.390923 4966 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:04:54.390978 4966 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:04:54.391098 4960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:54.391256 4960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:54.393291 4960 catalog_manager.cc:1347] Generated new cluster ID: 3da070455269474996d3436b134e2741
I20260619 12:04:54.393338 4960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:54.398638 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.386046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.0.203.190:38755)
0619 12:04:54.386254 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.386263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.386295 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:04:54.387034 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.387043 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.387107 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.387272 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:54.388201 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.389523 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.390677 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.391403 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.391526 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.394545 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:54.394549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:54.394550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:54.394563 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:54.396673 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.397224 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.397227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.397227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.397352 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.397590 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.397590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.397590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.397928 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:54.398095 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.398253 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.398412 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260619 12:04:54.402915 4960 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:54.403520 4960 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:54.411064 4960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9: Generated new TSK 0
I20260619 12:04:54.411304 4960 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:54.444252 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38755
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:54.534809 4972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:54.535153 4972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:54.535192 4972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:54.536533 4972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:54.536583 4972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:54.536623 4972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:54.538326 4972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:54.538607 4972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:54.538861 4972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:54.544224 4977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:54.544234 4978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:54.544344 4980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:54.544633 4972 server_base.cc:1034] running on GCE node
I20260619 12:04:54.544880 4972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:54.545163 4972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:54.546339 4972 hybrid_clock.cc:648] HybridClock initialized: now 1781870694546316 us; error 35 us; skew 500 ppm
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:54.549371 4972 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:54.549808 4972 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:54.549906 4972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:54.550053 4972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:54.551231 4972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "5cacd6b08945455d8e65e36c24de0e32"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "f8f84fdce1b004a57c33af9c297263c5"
server_key_iv: "a40c3fd3dd09a9cd63df74c71c950fa4"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.551626 4972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "5cacd6b08945455d8e65e36c24de0e32"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "f8f84fdce1b004a57c33af9c297263c5"
server_key_iv: "a40c3fd3dd09a9cd63df74c71c950fa4"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.552994 4972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:54.553831 4986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.554044 4972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:54.554153 4972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "5cacd6b08945455d8e65e36c24de0e32"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "f8f84fdce1b004a57c33af9c297263c5"
server_key_iv: "a40c3fd3dd09a9cd63df74c71c950fa4"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.554219 4972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:54.581403 4972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:54.583556 4972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:54.583698 4972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:54.583928 4972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:54.584254 4972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:54.584290 4972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.584319 4972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:54.584335 4972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.592790 4972 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:38509
I20260619 12:04:54.592859 5099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:38509 every 8 connection(s)
I20260619 12:04:54.593150 4972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:54.600550 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 4972
I20260619 12:04:54.600661 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:54.600880 814 external_mini_cluster.cc:1428] Setting key d2d265f6cb9a2e8f561985b6035849ef
I20260619 12:04:54.605460 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.594007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:44995 (local address 127.0.203.190:38755)
0619 12:04:54.594086 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.594088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.594293 (+ 205us) server_negotiation.cc:407] Connection header received
0619 12:04:54.594825 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.594826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.594839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.594932 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:54.596060 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.596641 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.598083 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.598326 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.598407 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.601202 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:54.601204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:54.601229 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:54.601241 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:54.603472 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.604128 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.604129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.604130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.604224 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.604629 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.604631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.604631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.604775 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:54.604845 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.605269 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.605388 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:04:54.605705 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.593943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38755 (local address 127.0.203.129:44995)
0619 12:04:54.594124 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:54.594171 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:54.594681 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:54.595009 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:54.595012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:54.595313 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:54.595909 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.595915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.596785 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.596787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.597946 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.597952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.598458 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.598459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.598614 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.599240 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:54.599249 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:54.601066 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:54.603625 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:54.603641 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:54.603644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:54.604033 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:54.604332 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:54.604333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:54.604333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:54.604513 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:54.604852 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:54.604858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:54.605077 (+ 219us) client_negotiation.cc:769] Sending connection context
0619 12:04:54.605209 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 12:04:54.605431 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260619 12:04:54.606489 5100 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38755
I20260619 12:04:54.606606 5100 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:54.606879 5100 heartbeater.cc:511] Master 127.0.203.190:38755 requested a full tablet report, sending...
I20260619 12:04:54.607635 4910 ts_manager.cc:194] Registered new tserver with Master: 5cacd6b08945455d8e65e36c24de0e32 (127.0.203.129:38509)
I20260619 12:04:54.608292 4910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:44995
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:54.628087 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38755
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:54.716578 5105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:54.716815 5105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:54.716854 5105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:54.718199 5105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:54.718256 5105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:54.718297 5105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:54.719974 5105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:54.720240 5105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:54.720472 5105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:54.725736 5111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:54.725813 5110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:54.725857 5105 server_base.cc:1034] running on GCE node
W20260619 12:04:54.725770 5113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:54.726238 5105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:54.726540 5105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:54.727692 5105 hybrid_clock.cc:648] HybridClock initialized: now 1781870694727671 us; error 46 us; skew 500 ppm
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:54.730589 5105 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:54.731029 5105 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:54.731083 5105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:54.731220 5105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:54.732543 5105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "9465006ec77d47dc9d95aa01e9bde7f4"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "5d67f0becb3db4ae38f35e0e9f9c2d5b"
server_key_iv: "a10343a4479f3da61e99d0c9903bac12"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.732909 5105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "9465006ec77d47dc9d95aa01e9bde7f4"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "5d67f0becb3db4ae38f35e0e9f9c2d5b"
server_key_iv: "a10343a4479f3da61e99d0c9903bac12"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.734402 5105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:54.735317 5119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.735595 5105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:54.735703 5105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "9465006ec77d47dc9d95aa01e9bde7f4"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "5d67f0becb3db4ae38f35e0e9f9c2d5b"
server_key_iv: "a10343a4479f3da61e99d0c9903bac12"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.735823 5105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:54.749364 5105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:54.752091 5105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:54.752254 5105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:54.752518 5105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:54.752908 5105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:54.752950 5105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.752985 5105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:54.753001 5105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.760296 5105 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:39455
I20260619 12:04:54.760385 5232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:39455 every 8 connection(s)
I20260619 12:04:54.760669 5105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:54.764097 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 5105
I20260619 12:04:54.764246 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:54.764480 814 external_mini_cluster.cc:1428] Setting key 774dda94e1179e8412d97424b5b60771
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:54.772535 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.761486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:50067 (local address 127.0.203.190:38755)
0619 12:04:54.761552 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.761554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.761825 (+ 271us) server_negotiation.cc:407] Connection header received
0619 12:04:54.762342 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.762344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.762357 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.762452 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:54.763537 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.764347 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.765669 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.766012 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.766100 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.768984 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:54.768986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:54.768986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:54.769002 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:54.770637 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.771240 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.771241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.771242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.771313 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.771676 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.771677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.771678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.771872 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:54.771952 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.772356 (+ 404us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.772452 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:04:54.772763 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.761436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38755 (local address 127.0.203.130:50067)
0619 12:04:54.761623 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:54.761672 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:54.762217 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:54.762527 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:54.762531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:54.762834 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:54.763397 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.763403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.764504 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.764505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.765575 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.765578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.766136 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.766137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.766279 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.766932 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:54.766942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:54.768843 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:54.770754 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:54.770765 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:54.770768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:54.771126 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:54.771426 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:54.771427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:54.771427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:54.771579 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:54.771956 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:54.771962 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:54.772173 (+ 211us) client_negotiation.cc:769] Sending connection context
0619 12:04:54.772309 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 12:04:54.772502 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260619 12:04:54.773533 5233 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38755
I20260619 12:04:54.773694 5233 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:54.773941 5233 heartbeater.cc:511] Master 127.0.203.190:38755 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
I20260619 12:04:54.774425 4910 ts_manager.cc:194] Registered new tserver with Master: 9465006ec77d47dc9d95aa01e9bde7f4 (127.0.203.130:39455)
I20260619 12:04:54.774869 4910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:50067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:54.794648 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38755
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:54.882134 5238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:54.882372 5238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:54.882401 5238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:54.883689 5238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:54.883744 5238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:54.883774 5238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:54.885433 5238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:54.885682 5238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:54.885881 5238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:54.891005 5244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:54.891094 5238 server_base.cc:1034] running on GCE node
W20260619 12:04:54.890993 5243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:54.890993 5246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:54.891516 5238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:54.891800 5238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:54.892944 5238 hybrid_clock.cc:648] HybridClock initialized: now 1781870694892916 us; error 47 us; skew 500 ppm
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:54.895720 5238 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:54.896085 5238 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:54.896131 5238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:54.896241 5238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:54.897485 5238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "c08a9a956e3b4ab4813db74ef9b71291"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "49e271fe4785af4cd117133efc8b4004"
server_key_iv: "a729a3ad9ae415a0006d74779ae0d026"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.897825 5238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "c08a9a956e3b4ab4813db74ef9b71291"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "49e271fe4785af4cd117133efc8b4004"
server_key_iv: "a729a3ad9ae415a0006d74779ae0d026"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.899152 5238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:04:54.899999 5252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.900197 5238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:04:54.900285 5238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "c08a9a956e3b4ab4813db74ef9b71291"
format_stamp: "Formatted at 2026-06-19 12:04:54 on dist-test-slave-44vj"
server_key: "49e271fe4785af4cd117133efc8b4004"
server_key_iv: "a729a3ad9ae415a0006d74779ae0d026"
server_key_version: "encryptionkey@0"
I20260619 12:04:54.900369 5238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:54.912402 5238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:54.914634 5238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:54.914772 5238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:54.915028 5238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:54.915392 5238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:54.915437 5238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.915462 5238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:54.915503 5238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:54.922755 5238 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:34653
I20260619 12:04:54.922857 5365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:34653 every 8 connection(s)
I20260619 12:04:54.923084 5238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:04:54.930269 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 5238
I20260619 12:04:54.930361 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:54.930529 814 external_mini_cluster.cc:1428] Setting key 63c85bd46daf8566fb3d3914d6a16a2e
Jun 19 12:04:54 dist-test-slave-44vj krb5kdc[4866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:54.933980 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.924196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:42289 (local address 127.0.203.190:38755)
0619 12:04:54.924248 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.924250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.925024 (+ 774us) server_negotiation.cc:407] Connection header received
0619 12:04:54.925561 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.925563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.925574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.925649 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:54.926652 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.927244 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.928434 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.928645 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.928731 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.931079 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:54.931080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:54.931080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:54.931089 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:54.932590 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.933031 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.933033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.933033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.933109 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.933330 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.933331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.933331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.933462 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:54.933518 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.933798 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.933884 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:04:54.934161 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.923860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38755 (local address 127.0.203.131:42289)
0619 12:04:54.924872 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:54.924919 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:54.925459 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:54.925713 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:54.925716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:54.926003 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:54.926519 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.926524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.927429 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.927430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.928322 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.928325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.928728 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.928729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.928837 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.929396 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:54.929406 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:54.930977 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:54.932693 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:54.932702 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:54.932705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:54.932945 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:54.933163 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:54.933164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:54.933164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:54.933276 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:54.933517 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:54.933524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:54.933676 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 12:04:54.933766 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 12:04:54.933932 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":96,"threads_started":1}
I20260619 12:04:54.934723 5366 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38755
I20260619 12:04:54.934816 5366 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:54.934953 5366 heartbeater.cc:511] Master 127.0.203.190:38755 requested a full tablet report, sending...
I20260619 12:04:54.935444 4910 ts_manager.cc:194] Registered new tserver with Master: c08a9a956e3b4ab4813db74ef9b71291 (127.0.203.131:34653)
I20260619 12:04:54.935916 4910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:42289
I20260619 12:04:54.942759 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:04:54.951892 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.944456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50082 (local address 127.0.203.190:38755)
0619 12:04:54.944542 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.944544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.944615 (+ 71us) server_negotiation.cc:407] Connection header received
0619 12:04:54.944672 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.944673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.944683 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.944741 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:54.945397 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.945920 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.946704 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.946907 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.946970 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.948397 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:54.948399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:54.948399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:54.948408 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:54.950070 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.950478 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.950480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.950480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.950552 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:54.950792 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:54.950793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:54.950793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:54.951017 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:54.951131 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.951442 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.951794 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:04:54.954180 4910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:54.956522 4910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:54.964460 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.960165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:34653 (local address 127.0.0.1:52434)
0619 12:04:54.960484 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:54.960493 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:54.960541 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:54.960801 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:54.960802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:54.960805 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:54.960915 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.960917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.961842 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.961844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.963058 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.963061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.964100 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.964101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.964189 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.964207 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 12:04:54.964291 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:04:54.964351 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":37,"thread_start_us":85,"threads_started":1}
I20260619 12:04:54.964717 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.959988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.0.203.130:39455)
0619 12:04:54.960207 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.960214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.960396 (+ 182us) server_negotiation.cc:407] Connection header received
0619 12:04:54.960548 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.960551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.960569 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.960627 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:54.960999 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.962194 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.963551 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.964243 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.964309 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.964360 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.964579 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.964634 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":101,"threads_started":1}
I20260619 12:04:54.964949 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.959835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:39455 (local address 127.0.0.1:60158)
0619 12:04:54.960319 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:54.960330 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:54.960432 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:54.960773 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:54.960777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:54.960780 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:54.960872 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.960875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.962333 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.962334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.963329 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.963331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.964397 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.964399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.964486 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.964492 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:04:54.964807 (+ 315us) client_negotiation.cc:240] Negotiation successful
0619 12:04:54.964875 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":4,"thread_start_us":117,"threads_started":1}
I20260619 12:04:54.964498 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.960391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.0.203.131:34653)
0619 12:04:54.960561 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.960565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.960576 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:04:54.960612 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.960613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.960650 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.960722 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:54.960995 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.961742 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.963170 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.963992 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.964077 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.964146 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.964325 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.964385 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
I20260619 12:04:54.965692 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.959812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:38509 (local address 127.0.0.1:49620)
0619 12:04:54.960197 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:54.960260 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:54.960355 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:54.960835 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:54.960836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:54.960839 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:54.960934 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.960936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.963555 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.963556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.964396 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:54.964398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.965426 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.965427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:54.965500 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.965506 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:04:54.965555 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:04:54.965607 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260619 12:04:54.965773 5167 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8ae5769494fbf957b56a4830960ce (DEFAULT_TABLE table=test-table [id=28702882c9b343dbb3289df57e5e9c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:54.965940 5379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.959919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.0.203.129:38509)
0619 12:04:54.960132 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:54.960137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:54.960400 (+ 263us) server_negotiation.cc:407] Connection header received
0619 12:04:54.960742 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:54.960744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:54.960759 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:54.960830 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:54.961011 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.962178 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.964522 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:54.965268 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:54.965329 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:54.965392 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:54.965823 (+ 431us) server_negotiation.cc:299] Negotiation successful
0619 12:04:54.965873 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260619 12:04:54.966106 5300 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8ae5769494fbf957b56a4830960ce (DEFAULT_TABLE table=test-table [id=28702882c9b343dbb3289df57e5e9c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:54.966208 5167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8ae5769494fbf957b56a4830960ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:54.966378 5300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8ae5769494fbf957b56a4830960ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:54.966538 5034 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8ae5769494fbf957b56a4830960ce (DEFAULT_TABLE table=test-table [id=28702882c9b343dbb3289df57e5e9c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:54.966832 5034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8ae5769494fbf957b56a4830960ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:54.969604 5383 tablet_bootstrap.cc:492] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32: Bootstrap starting.
I20260619 12:04:54.970103 5384 tablet_bootstrap.cc:492] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Bootstrap starting.
I20260619 12:04:54.970280 5383 tablet_bootstrap.cc:654] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:54.970497 5385 tablet_bootstrap.cc:492] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Bootstrap starting.
I20260619 12:04:54.970813 5384 tablet_bootstrap.cc:654] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:54.970847 5383 log.cc:824] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:54.971127 5385 tablet_bootstrap.cc:654] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:54.971557 5384 log.cc:824] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:54.971822 5383 tablet_bootstrap.cc:492] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32: No bootstrap required, opened a new log
I20260619 12:04:54.971815 5385 log.cc:824] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:54.971906 5383 ts_tablet_manager.cc:1397] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:04:54.972474 5384 tablet_bootstrap.cc:492] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: No bootstrap required, opened a new log
I20260619 12:04:54.972484 5385 tablet_bootstrap.cc:492] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: No bootstrap required, opened a new log
I20260619 12:04:54.972551 5384 ts_tablet_manager.cc:1397] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 12:04:54.972668 5385 ts_tablet_manager.cc:1397] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:04:54.973344 5383 raft_consensus.cc:348] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:54.973491 5383 raft_consensus.cc:374] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:54.973521 5383 raft_consensus.cc:729] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cacd6b08945455d8e65e36c24de0e32, State: Initialized, Role: FOLLOWER
I20260619 12:04:54.973661 5383 consensus_queue.cc:260] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:54.973938 5100 heartbeater.cc:503] Master 127.0.203.190:38755 was elected leader, sending a full tablet report...
I20260619 12:04:54.973953 5383 ts_tablet_manager.cc:1428] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:04:54.974690 5384 raft_consensus.cc:348] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:54.974690 5385 raft_consensus.cc:348] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:54.974879 5385 raft_consensus.cc:374] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:54.974879 5384 raft_consensus.cc:374] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:54.974921 5384 raft_consensus.cc:729] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9465006ec77d47dc9d95aa01e9bde7f4, State: Initialized, Role: FOLLOWER
I20260619 12:04:54.974921 5385 raft_consensus.cc:729] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08a9a956e3b4ab4813db74ef9b71291, State: Initialized, Role: FOLLOWER
I20260619 12:04:54.975044 5384 consensus_queue.cc:260] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:54.975061 5385 consensus_queue.cc:260] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:54.975405 5385 ts_tablet_manager.cc:1428] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 12:04:54.975402 5384 ts_tablet_manager.cc:1428] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:04:54.975425 5366 heartbeater.cc:503] Master 127.0.203.190:38755 was elected leader, sending a full tablet report...
I20260619 12:04:54.975425 5233 heartbeater.cc:503] Master 127.0.203.190:38755 was elected leader, sending a full tablet report...
I20260619 12:04:54.998999 5390 raft_consensus.cc:482] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:54.999138 5390 raft_consensus.cc:504] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:54.999559 5390 leader_election.cc:288] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cacd6b08945455d8e65e36c24de0e32 (127.0.203.129:38509), c08a9a956e3b4ab4813db74ef9b71291 (127.0.203.131:34653)
I20260619 12:04:55.002604 5379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.999904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:42193 (local address 127.0.203.129:38509)
0619 12:04:55.000001 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.000004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.000020 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:04:55.000061 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.000062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.000074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.000137 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:55.000325 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.000875 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.001889 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.002261 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.002326 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.002337 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.002471 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.002512 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 12:04:55.002604 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.999817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:38509 (local address 127.0.203.130:42193)
0619 12:04:54.999885 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:54.999897 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:54.999963 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.000163 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.000164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.000167 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:55.000261 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:55.000264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.000950 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.000952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:55.001802 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:55.001804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.002328 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.002329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:55.002412 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.002415 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.002460 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.002514 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":9}
I20260619 12:04:55.002717 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.000046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:45539 (local address 127.0.203.131:34653)
0619 12:04:55.000112 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.000115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.000243 (+ 128us) server_negotiation.cc:407] Connection header received
0619 12:04:55.000313 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.000314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.000324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.000374 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:04:55.000525 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.001009 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.001994 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.002377 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.002444 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.002455 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.002590 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.002637 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:04:55.002743 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:54.999946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:34653 (local address 127.0.203.130:45539)
0619 12:04:55.000180 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.000190 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.000249 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.000382 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.000383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.000385 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:04:55.000469 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:55.000471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.001085 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.001086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:55.001905 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:55.001907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.002457 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.002459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:55.002531 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.002535 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.002580 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.002630 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":151,"threads_started":1}
I20260619 12:04:55.003049 5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8ae5769494fbf957b56a4830960ce" candidate_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c08a9a956e3b4ab4813db74ef9b71291" is_pre_election: true
I20260619 12:04:55.003052 5054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8ae5769494fbf957b56a4830960ce" candidate_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cacd6b08945455d8e65e36c24de0e32" is_pre_election: true
I20260619 12:04:55.003240 5054 raft_consensus.cc:2454] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9465006ec77d47dc9d95aa01e9bde7f4 in term 0.
I20260619 12:04:55.003240 5320 raft_consensus.cc:2454] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9465006ec77d47dc9d95aa01e9bde7f4 in term 0.
I20260619 12:04:55.003533 5120 leader_election.cc:302] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9465006ec77d47dc9d95aa01e9bde7f4, c08a9a956e3b4ab4813db74ef9b71291; no voters:
I20260619 12:04:55.003723 5390 raft_consensus.cc:2781] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:55.003826 5390 raft_consensus.cc:482] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:55.003846 5390 raft_consensus.cc:3037] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.004617 5390 raft_consensus.cc:504] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:55.004758 5390 leader_election.cc:288] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [CANDIDATE]: Term 1 election: Requested vote from peers 5cacd6b08945455d8e65e36c24de0e32 (127.0.203.129:38509), c08a9a956e3b4ab4813db74ef9b71291 (127.0.203.131:34653)
I20260619 12:04:55.004956 5054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8ae5769494fbf957b56a4830960ce" candidate_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cacd6b08945455d8e65e36c24de0e32"
I20260619 12:04:55.004976 5320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8ae5769494fbf957b56a4830960ce" candidate_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c08a9a956e3b4ab4813db74ef9b71291"
I20260619 12:04:55.005040 5054 raft_consensus.cc:3037] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.005085 5320 raft_consensus.cc:3037] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.005820 5054 raft_consensus.cc:2454] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9465006ec77d47dc9d95aa01e9bde7f4 in term 1.
I20260619 12:04:55.006023 5120 leader_election.cc:302] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cacd6b08945455d8e65e36c24de0e32, 9465006ec77d47dc9d95aa01e9bde7f4; no voters:
I20260619 12:04:55.006088 5320 raft_consensus.cc:2454] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9465006ec77d47dc9d95aa01e9bde7f4 in term 1.
I20260619 12:04:55.006140 5390 raft_consensus.cc:2781] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:55.006325 5390 raft_consensus.cc:686] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 1 LEADER]: Becoming Leader. State: Replica: 9465006ec77d47dc9d95aa01e9bde7f4, State: Running, Role: LEADER
I20260619 12:04:55.006434 5390 consensus_queue.cc:237] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } }
I20260619 12:04:55.007477 4909 catalog_manager.cc:5495] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9465006ec77d47dc9d95aa01e9bde7f4 (127.0.203.130). New cstate: current_term: 1 leader_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9465006ec77d47dc9d95aa01e9bde7f4" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 39455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 } health_report { overall_health: UNKNOWN } } }
W20260619 12:04:55.011263 5234 tablet.cc:2369] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:04:55.023283 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.020822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60172 (local address 127.0.203.130:39455)
0619 12:04:55.020897 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.020899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.020908 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:04:55.020955 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.020956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.020978 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.021046 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:04:55.021200 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.021717 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.022519 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.022704 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.022812 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.022902 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:04:55.023035 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:04:55.023113 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.023135 (+ 22us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.023157 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:04:55.025300 5054 raft_consensus.cc:1261] T 2af8ae5769494fbf957b56a4830960ce P 5cacd6b08945455d8e65e36c24de0e32 [term 1 FOLLOWER]: Refusing update from remote peer 9465006ec77d47dc9d95aa01e9bde7f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:55.025379 5320 raft_consensus.cc:1261] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 1 FOLLOWER]: Refusing update from remote peer 9465006ec77d47dc9d95aa01e9bde7f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:55.025688 5393 consensus_queue.cc:1035] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cacd6b08945455d8e65e36c24de0e32" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:55.025835 5390 consensus_queue.cc:1035] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c08a9a956e3b4ab4813db74ef9b71291" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:55.029047 5397 mvcc.cc:205] Tried to move back new op lower bound from 7298542366820651008 to 7298542366747062272. Current Snapshot: MvccSnapshot[applied={T|T < 7298542366820651008}]
I20260619 12:04:55.029152 5398 mvcc.cc:205] Tried to move back new op lower bound from 7298542366820651008 to 7298542366747062272. Current Snapshot: MvccSnapshot[applied={T|T < 7298542366820651008}]
I20260619 12:04:55.031540 4909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:55.031685 4909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:55.032583 4909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e55731f17ac840118ae43499be6f40a9: Sending DeleteTablet for 3 replicas of tablet 2af8ae5769494fbf957b56a4830960ce
I20260619 12:04:55.032971 5167 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8ae5769494fbf957b56a4830960ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:55 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:60158
I20260619 12:04:55.032985 5034 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8ae5769494fbf957b56a4830960ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:55 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:49620
I20260619 12:04:55.033053 5300 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8ae5769494fbf957b56a4830960ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:55 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:52434
I20260619 12:04:55.033311 5407 tablet_replica.cc:330] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: stopping tablet replica
I20260619 12:04:55.033439 5407 raft_consensus.cc:2229] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:55.033569 5407 raft_consensus.cc:2258] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:55.033588 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4972
I20260619 12:04:55.034229 5406 tablet_replica.cc:330] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: stopping tablet replica
I20260619 12:04:55.034343 5406 raft_consensus.cc:2229] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:04:55.034456 5406 raft_consensus.cc:2258] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:55.034829 5407 ts_tablet_manager.cc:1905] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:55.035027 5406 ts_tablet_manager.cc:1905] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:55.037076 5406 ts_tablet_manager.cc:1918] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:04:55.037154 5406 log.cc:1196] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/2af8ae5769494fbf957b56a4830960ce
I20260619 12:04:55.037459 5406 ts_tablet_manager.cc:1939] T 2af8ae5769494fbf957b56a4830960ce P 9465006ec77d47dc9d95aa01e9bde7f4: Deleting consensus metadata
I20260619 12:04:55.037439 5407 ts_tablet_manager.cc:1918] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:04:55.037663 5407 log.cc:1196] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/2af8ae5769494fbf957b56a4830960ce
I20260619 12:04:55.037997 5407 ts_tablet_manager.cc:1939] T 2af8ae5769494fbf957b56a4830960ce P c08a9a956e3b4ab4813db74ef9b71291: Deleting consensus metadata
I20260619 12:04:55.037966 4896 catalog_manager.cc:4841] TS 9465006ec77d47dc9d95aa01e9bde7f4 (127.0.203.130:39455): tablet 2af8ae5769494fbf957b56a4830960ce (table test-table [id=28702882c9b343dbb3289df57e5e9c0d]) successfully deleted
I20260619 12:04:55.038581 4894 catalog_manager.cc:4841] TS c08a9a956e3b4ab4813db74ef9b71291 (127.0.203.131:34653): tablet 2af8ae5769494fbf957b56a4830960ce (table test-table [id=28702882c9b343dbb3289df57e5e9c0d]) successfully deleted
W20260619 12:04:55.040594 4894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:38509 (error 108)
I20260619 12:04:55.041204 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 5105
I20260619 12:04:55.041282 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.041149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:38509 (local address 127.0.0.1:49636)
0619 12:04:55.041208 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.041248 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:38509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:04:55.043919 4894 catalog_manager.cc:4584] TS 5cacd6b08945455d8e65e36c24de0e32 (127.0.203.129:38509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2af8ae5769494fbf957b56a4830960ce: Network error: Client connection negotiation failed: client connection to 127.0.203.129:38509: connect: Connection refused (error 111)
I20260619 12:04:55.045902 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 5238
I20260619 12:04:55.050424 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 4879
2026-06-19T12:04:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1914 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T12:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:55Z Disabled control of system clock
I20260619 12:04:55.080945 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43605
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:43605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:04:55.164181 5412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:55.164417 5412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:55.164440 5412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:55.165753 5412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:55.165828 5412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:55.165843 5412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:55.165854 5412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:55.165866 5412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:55.167449 5412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:43605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:55.167732 5412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:55.168047 5412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:55.173648 5420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.173663 5418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:55.173890 5412 server_base.cc:1034] running on GCE node
W20260619 12:04:55.173663 5417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:55.174273 5412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:55.174630 5412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:55.175813 5412 hybrid_clock.cc:648] HybridClock initialized: now 1781870695175802 us; error 43 us; skew 500 ppm
I20260619 12:04:55.177469 5412 webserver.cc:460] Webserver started at http://127.0.203.190:39787/ using document root <none> and password file <none>
I20260619 12:04:55.177701 5412 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:55.177752 5412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:55.177850 5412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:55.178982 5412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "6b2f8597df8744cf8e57d06592290f04"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "421fd3f5418078798a8b569e257d2dca"
server_key_iv: "18602c46b5a7e595fad109e6f5eb711a"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.179307 5412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "6b2f8597df8744cf8e57d06592290f04"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "421fd3f5418078798a8b569e257d2dca"
server_key_iv: "18602c46b5a7e595fad109e6f5eb711a"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.180711 5412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:55.181586 5426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.181847 5412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:55.181941 5412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "6b2f8597df8744cf8e57d06592290f04"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "421fd3f5418078798a8b569e257d2dca"
server_key_iv: "18602c46b5a7e595fad109e6f5eb711a"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.182045 5412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:55.186342 5412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:55.186664 5412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:55.186796 5412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:55.191166 5412 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:43605
I20260619 12:04:55.191246 5478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:43605 every 8 connection(s)
I20260619 12:04:55.191601 5412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:55.192402 5479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:55.195755 5479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04: Bootstrap starting.
I20260619 12:04:55.196448 5479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:55.196444 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 5412
I20260619 12:04:55.196537 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:55.196743 814 external_mini_cluster.cc:1428] Setting key 6835f9df6baa5253a0a17cb40f5707e0
I20260619 12:04:55.196768 5479 log.cc:824] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:55.197774 5479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04: No bootstrap required, opened a new log
I20260619 12:04:55.199805 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.197399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.0.203.190:43605)
0619 12:04:55.197644 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.197655 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.197686 (+ 31us) server_negotiation.cc:407] Connection header received
0619 12:04:55.198309 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.198317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.198364 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.198518 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.199311 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.199315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.199315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.199321 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.199391 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.199408 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.199487 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.199541 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.199617 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260619 12:04:55.200040 5479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2f8597df8744cf8e57d06592290f04" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43605 } }
I20260619 12:04:55.200302 5479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:55.200345 5479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2f8597df8744cf8e57d06592290f04, State: Initialized, Role: FOLLOWER
I20260619 12:04:55.200484 5479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2f8597df8744cf8e57d06592290f04" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43605 } }
I20260619 12:04:55.200615 5479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:55.200652 5479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:55.200706 5479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.201720 5479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2f8597df8744cf8e57d06592290f04" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43605 } }
I20260619 12:04:55.201831 5479 leader_election.cc:302] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b2f8597df8744cf8e57d06592290f04; no voters:
I20260619 12:04:55.202010 5479 leader_election.cc:288] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:55.202112 5484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:55.202322 5479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:55.202342 5484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [term 1 LEADER]: Becoming Leader. State: Replica: 6b2f8597df8744cf8e57d06592290f04, State: Running, Role: LEADER
I20260619 12:04:55.202508 5484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2f8597df8744cf8e57d06592290f04" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43605 } }
I20260619 12:04:55.202960 5486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b2f8597df8744cf8e57d06592290f04. Latest consensus state: current_term: 1 leader_uuid: "6b2f8597df8744cf8e57d06592290f04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2f8597df8744cf8e57d06592290f04" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43605 } } }
I20260619 12:04:55.203070 5486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:55.202965 5485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b2f8597df8744cf8e57d06592290f04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2f8597df8744cf8e57d06592290f04" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43605 } } }
I20260619 12:04:55.203287 5485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:55.203682 5493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:55.204353 5493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:55.206396 5493 catalog_manager.cc:1347] Generated new cluster ID: 0cd2a4e70ec340d79a6a4130f1e3db4a
I20260619 12:04:55.206467 5493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:55.219710 5493 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:55.219957 5493 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:55.231186 5493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04: Generated new TSK 0
I20260619 12:04:55.231451 5493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:04:55.241216 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43605
--builtin_ntp_servers=127.0.203.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:04:55.326454 5503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:55.326680 5503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:55.326705 5503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:55.328027 5503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:55.328083 5503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:55.328110 5503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:55.329687 5503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:55.329952 5503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:55.330268 5503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:55.335253 5509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.335305 5508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.335397 5511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:55.335705 5503 server_base.cc:1034] running on GCE node
I20260619 12:04:55.335971 5503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:55.336289 5503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:55.337460 5503 hybrid_clock.cc:648] HybridClock initialized: now 1781870695337435 us; error 44 us; skew 500 ppm
I20260619 12:04:55.339134 5503 webserver.cc:460] Webserver started at http://127.0.203.129:33723/ using document root <none> and password file <none>
I20260619 12:04:55.339464 5503 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:55.339522 5503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:55.339674 5503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:55.340966 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "28b73ccc2361483c8078d144e014cc21"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "dc685ed8276bfb7cb0e72b2e3ca6f707"
server_key_iv: "1bd9b0d7688682f3266821cde3d6f418"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.341351 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "28b73ccc2361483c8078d144e014cc21"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "dc685ed8276bfb7cb0e72b2e3ca6f707"
server_key_iv: "1bd9b0d7688682f3266821cde3d6f418"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.342934 5503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:55.344069 5517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.344343 5503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:04:55.344445 5503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "28b73ccc2361483c8078d144e014cc21"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "dc685ed8276bfb7cb0e72b2e3ca6f707"
server_key_iv: "1bd9b0d7688682f3266821cde3d6f418"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.344525 5503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:55.348246 5503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:55.348580 5503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:55.348711 5503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:55.348937 5503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:55.349277 5503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:55.349321 5503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.349351 5503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:55.349372 5503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.356446 5503 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:44933
I20260619 12:04:55.356501 5630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:44933 every 8 connection(s)
I20260619 12:04:55.356884 5503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:04:55.360225 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.357751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:35749 (local address 127.0.203.190:43605)
0619 12:04:55.357858 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.357861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.358094 (+ 233us) server_negotiation.cc:407] Connection header received
0619 12:04:55.358629 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.358630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.358639 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.358707 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.359988 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.359990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.359990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.359995 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.360013 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.360018 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.360076 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.360143 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.360157 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 12:04:55.360400 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.357678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43605 (local address 127.0.203.129:35749)
0619 12:04:55.357904 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.357974 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.358513 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.358783 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.358787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.358796 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.359818 (+ 1022us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.359819 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.359853 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.359886 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.360075 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.360078 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.360081 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.360142 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.360200 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
I20260619 12:04:55.360931 5631 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43605
I20260619 12:04:55.361033 5631 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:55.361168 5631 heartbeater.cc:511] Master 127.0.203.190:43605 requested a full tablet report, sending...
I20260619 12:04:55.361723 5443 ts_manager.cc:194] Registered new tserver with Master: 28b73ccc2361483c8078d144e014cc21 (127.0.203.129:44933)
I20260619 12:04:55.366269 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 5503
I20260619 12:04:55.366374 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:55.366554 814 external_mini_cluster.cc:1428] Setting key f64274f20d41d1569acd0104168cdd2d
I20260619 12:04:55.367759 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43605
--builtin_ntp_servers=127.0.203.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:04:55.458557 5634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:55.458824 5634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:55.458859 5634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:55.460286 5634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:55.460340 5634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:55.460381 5634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:55.461875 5634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:55.462149 5634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:55.462481 5634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:55.467931 5640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.467942 5639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.468284 5642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:55.468924 5634 server_base.cc:1034] running on GCE node
I20260619 12:04:55.469225 5634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:55.469566 5634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:55.470719 5634 hybrid_clock.cc:648] HybridClock initialized: now 1781870695470706 us; error 30 us; skew 500 ppm
I20260619 12:04:55.472294 5634 webserver.cc:460] Webserver started at http://127.0.203.130:37945/ using document root <none> and password file <none>
I20260619 12:04:55.472590 5634 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:55.472651 5634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:55.472808 5634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:55.474074 5634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "8bab4aa0ea01448a9ad086971d2a6bb9"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "596ffebd330bb9d1073e3c6c3504f62d"
server_key_iv: "f61cf37b5c8a0ed90e4171f018fd1f58"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.474447 5634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "8bab4aa0ea01448a9ad086971d2a6bb9"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "596ffebd330bb9d1073e3c6c3504f62d"
server_key_iv: "f61cf37b5c8a0ed90e4171f018fd1f58"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.476405 5634 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 12:04:55.477363 5648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.477638 5634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:55.477749 5634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "8bab4aa0ea01448a9ad086971d2a6bb9"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "596ffebd330bb9d1073e3c6c3504f62d"
server_key_iv: "f61cf37b5c8a0ed90e4171f018fd1f58"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.477825 5634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:55.481684 5634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:55.481999 5634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:55.482124 5634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:55.482336 5634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:55.482652 5634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:55.482686 5634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.482726 5634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:55.482751 5634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.489809 5634 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:35587
I20260619 12:04:55.489881 5761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:35587 every 8 connection(s)
I20260619 12:04:55.490250 5634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:55.494084 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 5634
I20260619 12:04:55.494112 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.491233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:60971 (local address 127.0.203.190:43605)
0619 12:04:55.491352 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.491354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.491624 (+ 270us) server_negotiation.cc:407] Connection header received
0619 12:04:55.492196 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.492197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.492206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.492301 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.493751 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.493753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.493753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.493758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.493778 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.493782 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.493848 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.494001 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.494023 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 12:04:55.494184 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:55.494334 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.491187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43605 (local address 127.0.203.130:60971)
0619 12:04:55.491473 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.491519 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.492067 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.492441 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.492445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.492456 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.493514 (+ 1058us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.493515 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.493549 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.493591 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.493855 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.493858 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.493862 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.493961 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.494052 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":127,"threads_started":1}
I20260619 12:04:55.494393 814 external_mini_cluster.cc:1428] Setting key 7345d497192193fb2d1416461f2edc07
I20260619 12:04:55.494874 5762 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43605
I20260619 12:04:55.494975 5762 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:55.495164 5762 heartbeater.cc:511] Master 127.0.203.190:43605 requested a full tablet report, sending...
I20260619 12:04:55.495589 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43605
--builtin_ntp_servers=127.0.203.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 12:04:55.495608 5443 ts_manager.cc:194] Registered new tserver with Master: 8bab4aa0ea01448a9ad086971d2a6bb9 (127.0.203.130:35587)
W20260619 12:04:55.582392 5765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:55.582635 5765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:55.582659 5765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:55.583974 5765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:55.584028 5765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:55.584056 5765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:55.585559 5765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:55.585814 5765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:55.586131 5765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:55.591593 5773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.591555 5770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.591609 5771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:55.591850 5765 server_base.cc:1034] running on GCE node
I20260619 12:04:55.592199 5765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:55.592494 5765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:55.593685 5765 hybrid_clock.cc:648] HybridClock initialized: now 1781870695593667 us; error 40 us; skew 500 ppm
I20260619 12:04:55.595270 5765 webserver.cc:460] Webserver started at http://127.0.203.131:41423/ using document root <none> and password file <none>
I20260619 12:04:55.595597 5765 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:55.595657 5765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:55.595811 5765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:55.596984 5765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "cdc2605763904bc292f9fd605882cbf6"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "7830afa88b870f30fb12e0ec6bb480e5"
server_key_iv: "877ff7d68019c5e849517923c1008b4b"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.597350 5765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "cdc2605763904bc292f9fd605882cbf6"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "7830afa88b870f30fb12e0ec6bb480e5"
server_key_iv: "877ff7d68019c5e849517923c1008b4b"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.598728 5765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:04:55.599592 5779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.599782 5765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:55.599879 5765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "cdc2605763904bc292f9fd605882cbf6"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "7830afa88b870f30fb12e0ec6bb480e5"
server_key_iv: "877ff7d68019c5e849517923c1008b4b"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.599943 5765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:55.603587 5765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:55.603871 5765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:55.603998 5765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:55.604216 5765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:55.604525 5765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:55.604557 5765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.604585 5765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:55.604602 5765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.610704 5765 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:40439
I20260619 12:04:55.610759 5892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:40439 every 8 connection(s)
I20260619 12:04:55.611101 5765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:04:55.614296 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.611959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:37355 (local address 127.0.203.190:43605)
0619 12:04:55.612013 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.612015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.612340 (+ 325us) server_negotiation.cc:407] Connection header received
0619 12:04:55.612816 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.612818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.612824 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.612878 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.614043 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.614045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.614045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.614050 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.614066 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.614071 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.614123 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.614209 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.614223 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:04:55.614493 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.611964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43605 (local address 127.0.203.131:37355)
0619 12:04:55.612199 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.612241 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.612725 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.612954 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.612957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.612967 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.613895 (+ 928us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.613895 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.613926 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.613954 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.614126 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.614131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.614134 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.614216 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.614282 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260619 12:04:55.615048 5893 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43605
I20260619 12:04:55.615131 5893 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:55.615248 5893 heartbeater.cc:511] Master 127.0.203.190:43605 requested a full tablet report, sending...
I20260619 12:04:55.615664 5443 ts_manager.cc:194] Registered new tserver with Master: cdc2605763904bc292f9fd605882cbf6 (127.0.203.131:40439)
I20260619 12:04:55.621326 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 5765
I20260619 12:04:55.621433 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:55.621574 814 external_mini_cluster.cc:1428] Setting key 521a8582a1ad251ad138cac6419eaacf
I20260619 12:04:55.622679 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:04:55.625358 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.624197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.0.203.190:43605)
0619 12:04:55.624248 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.624250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.624400 (+ 150us) server_negotiation.cc:407] Connection header received
0619 12:04:55.624467 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.624468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.624474 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.624520 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.625143 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.625144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.625144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.625155 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.625169 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.625173 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.625222 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.625275 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.625287 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:04:55.626873 5443 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:04:55.628544 5443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:04:55.632296 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.630727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35587 (local address 127.0.0.1:57250)
0619 12:04:55.630916 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.630963 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.631032 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.631192 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.631196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.631204 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.631797 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.631797 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.631819 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.631849 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.632129 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.632137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.632140 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.632209 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.632225 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260619 12:04:55.632328 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.630830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.0.203.130:35587)
0619 12:04:55.631036 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.631041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.631056 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:04:55.631086 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.631089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.631101 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.631177 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.631939 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.631951 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.631951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.631957 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.632003 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.632032 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.632101 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.632235 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.632252 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260619 12:04:55.632905 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.630976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:40439 (local address 127.0.0.1:50034)
0619 12:04:55.631606 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.631616 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.631666 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.631978 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.631979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.631982 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.632499 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.632500 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.632508 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.632521 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.632757 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.632758 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.632759 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.632786 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.632799 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":71,"threads_started":1}
I20260619 12:04:55.632969 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.631402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.0.203.131:40439)
0619 12:04:55.631816 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.631820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.631830 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:04:55.631857 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.631859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.631868 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.631973 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.632607 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.632614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.632614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.632618 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.632655 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.632686 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.632863 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.632881 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.632894 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":79,"threads_started":1}
I20260619 12:04:55.633092 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.631131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45902 (local address 127.0.203.129:44933)
0619 12:04:55.631438 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.631443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.631455 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:04:55.631484 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.631486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.631496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.631568 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.632568 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.632574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.632575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.632579 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.632609 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.632627 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.632688 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.632999 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.633016 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":87,"threads_started":1}
I20260619 12:04:55.632824 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.630729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:44933 (local address 127.0.0.1:45902)
0619 12:04:55.631113 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.631125 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.631200 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.631906 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.631907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.631911 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.632462 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.632463 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.632471 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.632486 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.632710 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.632711 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.632711 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.632751 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.632763 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":119,"threads_started":1}
I20260619 12:04:55.633065 5696 tablet_service.cc:1467] Processing CreateTablet for tablet e3d26da3b200407ea164a6159051500a (DEFAULT_TABLE table=test-table [id=417d8fe9ebed421dad86d9746235ca78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:55.633446 5696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3d26da3b200407ea164a6159051500a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:55.633968 5565 tablet_service.cc:1467] Processing CreateTablet for tablet e3d26da3b200407ea164a6159051500a (DEFAULT_TABLE table=test-table [id=417d8fe9ebed421dad86d9746235ca78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:55.634032 5827 tablet_service.cc:1467] Processing CreateTablet for tablet e3d26da3b200407ea164a6159051500a (DEFAULT_TABLE table=test-table [id=417d8fe9ebed421dad86d9746235ca78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:04:55.634335 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3d26da3b200407ea164a6159051500a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:55.634352 5827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3d26da3b200407ea164a6159051500a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:55.636878 5907 tablet_bootstrap.cc:492] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Bootstrap starting.
I20260619 12:04:55.637553 5909 tablet_bootstrap.cc:492] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Bootstrap starting.
I20260619 12:04:55.637995 5907 tablet_bootstrap.cc:654] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:55.638120 5909 tablet_bootstrap.cc:654] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:55.638303 5907 log.cc:824] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:55.638414 5909 log.cc:824] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:55.638867 5908 tablet_bootstrap.cc:492] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21: Bootstrap starting.
I20260619 12:04:55.639117 5907 tablet_bootstrap.cc:492] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: No bootstrap required, opened a new log
I20260619 12:04:55.639173 5909 tablet_bootstrap.cc:492] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: No bootstrap required, opened a new log
I20260619 12:04:55.639226 5909 ts_tablet_manager.cc:1397] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:04:55.639532 5907 ts_tablet_manager.cc:1397] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:04:55.639557 5908 tablet_bootstrap.cc:654] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:55.639878 5908 log.cc:824] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:55.640553 5908 tablet_bootstrap.cc:492] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21: No bootstrap required, opened a new log
I20260619 12:04:55.640615 5908 ts_tablet_manager.cc:1397] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:04:55.641925 5909 raft_consensus.cc:348] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.642014 5907 raft_consensus.cc:348] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.642143 5909 raft_consensus.cc:374] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:55.642190 5909 raft_consensus.cc:729] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc2605763904bc292f9fd605882cbf6, State: Initialized, Role: FOLLOWER
I20260619 12:04:55.642321 5909 consensus_queue.cc:260] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.642143 5907 raft_consensus.cc:374] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:55.642550 5907 raft_consensus.cc:729] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bab4aa0ea01448a9ad086971d2a6bb9, State: Initialized, Role: FOLLOWER
I20260619 12:04:55.642661 5909 ts_tablet_manager.cc:1428] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 12:04:55.642781 5893 heartbeater.cc:503] Master 127.0.203.190:43605 was elected leader, sending a full tablet report...
I20260619 12:04:55.642793 5907 consensus_queue.cc:260] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.642699 5908 raft_consensus.cc:348] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.642889 5908 raft_consensus.cc:374] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:55.642927 5908 raft_consensus.cc:729] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b73ccc2361483c8078d144e014cc21, State: Initialized, Role: FOLLOWER
I20260619 12:04:55.643064 5907 ts_tablet_manager.cc:1428] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:55.643160 5762 heartbeater.cc:503] Master 127.0.203.190:43605 was elected leader, sending a full tablet report...
I20260619 12:04:55.643167 5908 consensus_queue.cc:260] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.643462 5908 ts_tablet_manager.cc:1428] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:04:55.643535 5631 heartbeater.cc:503] Master 127.0.203.190:43605 was elected leader, sending a full tablet report...
I20260619 12:04:55.677706 5915 raft_consensus.cc:482] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:04:55.677888 5915 raft_consensus.cc:504] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.678296 5915 leader_election.cc:288] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bab4aa0ea01448a9ad086971d2a6bb9 (127.0.203.130:35587), cdc2605763904bc292f9fd605882cbf6 (127.0.203.131:40439)
I20260619 12:04:55.680043 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.678705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:57893 (local address 127.0.203.130:35587)
0619 12:04:55.678758 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.678760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.678774 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:04:55.678860 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.678862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.678869 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.678991 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.679757 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.679759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.679760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.679765 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.679791 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.679797 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.679862 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.679937 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.679952 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:04:55.680064 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.678994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:53847 (local address 127.0.203.131:40439)
0619 12:04:55.679051 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.679053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.679199 (+ 146us) server_negotiation.cc:407] Connection header received
0619 12:04:55.679240 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.679241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.679273 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.679350 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.679815 (+ 465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.679817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.679818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.679823 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.679845 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.679850 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.679905 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.679988 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.680003 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:04:55.680043 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.678638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:35587 (local address 127.0.203.129:57893)
0619 12:04:55.678708 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.678724 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.678781 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.679068 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.679070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.679075 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.679629 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.679630 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.679637 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.679649 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.679874 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.679876 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.679878 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.679932 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.679952 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 12:04:55.680073 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.678857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:40439 (local address 127.0.203.129:53847)
0619 12:04:55.679090 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.679102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.679167 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.679388 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.679389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.679392 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:55.679728 (+ 336us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:04:55.679729 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:55.679733 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:04:55.679742 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:55.679906 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:55.679907 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:55.679908 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:04:55.679950 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 12:04:55.679967 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":144,"threads_started":1}
I20260619 12:04:55.680485 5716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3d26da3b200407ea164a6159051500a" candidate_uuid: "28b73ccc2361483c8078d144e014cc21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" is_pre_election: true
I20260619 12:04:55.680649 5716 raft_consensus.cc:2454] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b73ccc2361483c8078d144e014cc21 in term 0.
I20260619 12:04:55.680629 5847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3d26da3b200407ea164a6159051500a" candidate_uuid: "28b73ccc2361483c8078d144e014cc21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdc2605763904bc292f9fd605882cbf6" is_pre_election: true
I20260619 12:04:55.680792 5847 raft_consensus.cc:2454] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b73ccc2361483c8078d144e014cc21 in term 0.
I20260619 12:04:55.680929 5521 leader_election.cc:302] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b73ccc2361483c8078d144e014cc21, 8bab4aa0ea01448a9ad086971d2a6bb9; no voters:
I20260619 12:04:55.681069 5915 raft_consensus.cc:2781] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:04:55.681131 5915 raft_consensus.cc:482] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:04:55.681155 5915 raft_consensus.cc:3037] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.682168 5915 raft_consensus.cc:504] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.682310 5915 leader_election.cc:288] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [CANDIDATE]: Term 1 election: Requested vote from peers 8bab4aa0ea01448a9ad086971d2a6bb9 (127.0.203.130:35587), cdc2605763904bc292f9fd605882cbf6 (127.0.203.131:40439)
I20260619 12:04:55.682538 5847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3d26da3b200407ea164a6159051500a" candidate_uuid: "28b73ccc2361483c8078d144e014cc21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdc2605763904bc292f9fd605882cbf6"
I20260619 12:04:55.682523 5716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3d26da3b200407ea164a6159051500a" candidate_uuid: "28b73ccc2361483c8078d144e014cc21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9"
I20260619 12:04:55.682626 5847 raft_consensus.cc:3037] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.682626 5716 raft_consensus.cc:3037] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.683637 5716 raft_consensus.cc:2454] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28b73ccc2361483c8078d144e014cc21 in term 1.
I20260619 12:04:55.683637 5847 raft_consensus.cc:2454] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28b73ccc2361483c8078d144e014cc21 in term 1.
I20260619 12:04:55.683904 5521 leader_election.cc:302] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b73ccc2361483c8078d144e014cc21, 8bab4aa0ea01448a9ad086971d2a6bb9; no voters:
I20260619 12:04:55.683998 5915 raft_consensus.cc:2781] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:55.684203 5915 raft_consensus.cc:686] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [term 1 LEADER]: Becoming Leader. State: Replica: 28b73ccc2361483c8078d144e014cc21, State: Running, Role: LEADER
I20260619 12:04:55.684315 5915 consensus_queue.cc:237] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } }
I20260619 12:04:55.685367 5442 catalog_manager.cc:5495] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28b73ccc2361483c8078d144e014cc21 (127.0.203.129). New cstate: current_term: 1 leader_uuid: "28b73ccc2361483c8078d144e014cc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b73ccc2361483c8078d144e014cc21" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 44933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 } health_report { overall_health: UNKNOWN } } }
I20260619 12:04:55.697765 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.696677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.0.203.129:44933)
0619 12:04:55.696732 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.696734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.696742 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:04:55.696776 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.696777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.696789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.696836 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.697493 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.697494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.697494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.697500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.697516 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.697522 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.697587 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.697678 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.697692 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:04:55.700022 5847 raft_consensus.cc:1261] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 1 FOLLOWER]: Refusing update from remote peer 28b73ccc2361483c8078d144e014cc21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:55.700038 5716 raft_consensus.cc:1261] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 1 FOLLOWER]: Refusing update from remote peer 28b73ccc2361483c8078d144e014cc21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:04:55.700408 5915 consensus_queue.cc:1035] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdc2605763904bc292f9fd605882cbf6" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:55.700567 5917 consensus_queue.cc:1035] T e3d26da3b200407ea164a6159051500a P 28b73ccc2361483c8078d144e014cc21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bab4aa0ea01448a9ad086971d2a6bb9" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 35587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:04:55.704182 5922 mvcc.cc:205] Tried to move back new op lower bound from 7298542369584070656 to 7298542369523691520. Current Snapshot: MvccSnapshot[applied={T|T < 7298542369584070656}]
I20260619 12:04:55.704802 5921 mvcc.cc:205] Tried to move back new op lower bound from 7298542369584070656 to 7298542369523691520. Current Snapshot: MvccSnapshot[applied={T|T < 7298542369584070656 or (T in {7298542369584070656})}]
I20260619 12:04:55.706274 5442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:55.706404 5442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:04:55.707449 5442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b2f8597df8744cf8e57d06592290f04: Sending DeleteTablet for 3 replicas of tablet e3d26da3b200407ea164a6159051500a
I20260619 12:04:55.707808 5565 tablet_service.cc:1514] Processing DeleteTablet for tablet e3d26da3b200407ea164a6159051500a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:55 UTC) from {username='slave'} at 127.0.0.1:45902
I20260619 12:04:55.707907 5827 tablet_service.cc:1514] Processing DeleteTablet for tablet e3d26da3b200407ea164a6159051500a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:55 UTC) from {username='slave'} at 127.0.0.1:50034
I20260619 12:04:55.707885 5696 tablet_service.cc:1514] Processing DeleteTablet for tablet e3d26da3b200407ea164a6159051500a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:04:55 UTC) from {username='slave'} at 127.0.0.1:57250
I20260619 12:04:55.708186 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 5503
I20260619 12:04:55.709052 5932 tablet_replica.cc:330] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: stopping tablet replica
I20260619 12:04:55.709154 5932 raft_consensus.cc:2229] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:55.709247 5932 raft_consensus.cc:2258] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:55.709401 5931 tablet_replica.cc:330] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: stopping tablet replica
I20260619 12:04:55.709491 5931 raft_consensus.cc:2229] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:04:55.709574 5931 raft_consensus.cc:2258] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:04:55.710011 5932 ts_tablet_manager.cc:1905] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:55.710172 5931 ts_tablet_manager.cc:1905] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:04:55.711561 5932 ts_tablet_manager.cc:1918] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:04:55.711624 5932 log.cc:1196] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/e3d26da3b200407ea164a6159051500a
I20260619 12:04:55.711848 5932 ts_tablet_manager.cc:1939] T e3d26da3b200407ea164a6159051500a P cdc2605763904bc292f9fd605882cbf6: Deleting consensus metadata
I20260619 12:04:55.712225 5931 ts_tablet_manager.cc:1918] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:04:55.712265 5430 catalog_manager.cc:4841] TS cdc2605763904bc292f9fd605882cbf6 (127.0.203.131:40439): tablet e3d26da3b200407ea164a6159051500a (table test-table [id=417d8fe9ebed421dad86d9746235ca78]) successfully deleted
I20260619 12:04:55.712360 5931 log.cc:1196] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/e3d26da3b200407ea164a6159051500a
I20260619 12:04:55.712642 5931 ts_tablet_manager.cc:1939] T e3d26da3b200407ea164a6159051500a P 8bab4aa0ea01448a9ad086971d2a6bb9: Deleting consensus metadata
I20260619 12:04:55.713097 5430 catalog_manager.cc:4841] TS 8bab4aa0ea01448a9ad086971d2a6bb9 (127.0.203.130:35587): tablet e3d26da3b200407ea164a6159051500a (table test-table [id=417d8fe9ebed421dad86d9746235ca78]) successfully deleted
W20260619 12:04:55.714335 5429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:44933 (error 108)
I20260619 12:04:55.714450 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 5634
I20260619 12:04:55.715269 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.715134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:44933 (local address 127.0.0.1:45908)
0619 12:04:55.715183 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.715230 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:44933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 12:04:55.715397 5429 catalog_manager.cc:4584] TS 28b73ccc2361483c8078d144e014cc21 (127.0.203.129:44933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3d26da3b200407ea164a6159051500a: Network error: Client connection negotiation failed: client connection to 127.0.203.129:44933: connect: Connection refused (error 111)
I20260619 12:04:55.719135 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 5765
I20260619 12:04:55.724069 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 5412
2026-06-19T12:04:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (672 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 12:04:55.744105 814 mini_oidc.cc:152] Starting JWKS server
I20260619 12:04:55.745350 814 webserver.cc:460] Webserver started at https://127.0.0.1:36309/ using document root <none> and password file <none>
I20260619 12:04:55.745463 814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:04:55.745630 814 webserver.cc:460] Webserver started at http://127.0.0.1:35633/ using document root <none> and password file <none>
2026-06-19T12:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:55Z Disabled control of system clock
I20260619 12:04:55.755286 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36637
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:36733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:36637
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36309/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/testchainca.cert with env {}
W20260619 12:04:55.845999 5939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:55.846269 5939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:55.846292 5939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:55.847666 5939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:55.847718 5939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:55.847729 5939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:55.847749 5939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:55.847761 5939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:04:55.847790 5939 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:04:55.847805 5939 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36309/jwks.json
I20260619 12:04:55.849560 5939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:36637
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36309/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:55.849858 5939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:55.850103 5939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:55.855377 5945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:55.855389 5947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:55.855484 5939 server_base.cc:1034] running on GCE node
W20260619 12:04:55.855733 5944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:55.856098 5939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:55.856452 5939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:55.857631 5939 hybrid_clock.cc:648] HybridClock initialized: now 1781870695857602 us; error 54 us; skew 500 ppm
I20260619 12:04:55.859151 5939 webserver.cc:460] Webserver started at http://127.0.203.190:42257/ using document root <none> and password file <none>
I20260619 12:04:55.859442 5939 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:55.859501 5939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:55.859632 5939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:55.860808 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "d693995c16b36d0abae75ffeebbc069f"
server_key_iv: "786957ec8338cd4ea832c3a60841a9f8"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.861167 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "d693995c16b36d0abae75ffeebbc069f"
server_key_iv: "786957ec8338cd4ea832c3a60841a9f8"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.862561 5939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:55.863493 5953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:55.863705 5939 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:55.863806 5939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1"
format_stamp: "Formatted at 2026-06-19 12:04:55 on dist-test-slave-44vj"
server_key: "d693995c16b36d0abae75ffeebbc069f"
server_key_iv: "786957ec8338cd4ea832c3a60841a9f8"
server_key_version: "encryptionkey@0"
I20260619 12:04:55.863883 5939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:55.893697 5939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:55.894047 5939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:55.894186 5939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:55.899158 5939 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:36637
I20260619 12:04:55.899214 6005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:36637 every 8 connection(s)
I20260619 12:04:55.899659 5939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:55.900868 6006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:55.901489 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 5939
I20260619 12:04:55.901592 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:55.901836 814 external_mini_cluster.cc:1428] Setting key fcb9b3763c99472090cd75d4c1962cb5
I20260619 12:04:55.904587 6006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1: Bootstrap starting.
I20260619 12:04:55.905345 6006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:55.905679 6006 log.cc:824] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:55.906606 6006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1: No bootstrap required, opened a new log
I20260619 12:04:55.908263 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.902597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.0.203.190:36637)
0619 12:04:55.902881 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.902890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.902922 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:04:55.903635 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.903645 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.903723 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.903875 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:55.904200 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.905187 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.906238 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.906717 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.906872 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.907575 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:55.907580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:55.907581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:55.907790 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:55.907829 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:55.907842 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:55.907899 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:55.907973 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 12:04:55.908068 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":113,"threads_started":1}
I20260619 12:04:55.909133 6006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36637 } }
I20260619 12:04:55.909339 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:04:55.909359 6006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:55.909392 6006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebc6fe1ae8f48cb95d2fd34899f85f1, State: Initialized, Role: FOLLOWER
I20260619 12:04:55.909545 6006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36637 } }
I20260619 12:04:55.909629 6006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:55.909654 6006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:55.909709 6006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:55.910526 6006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36637 } }
I20260619 12:04:55.910667 6006 leader_election.cc:302] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ebc6fe1ae8f48cb95d2fd34899f85f1; no voters:
I20260619 12:04:55.910890 6006 leader_election.cc:288] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:55.910969 6011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:55.911178 6011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [term 1 LEADER]: Becoming Leader. State: Replica: 4ebc6fe1ae8f48cb95d2fd34899f85f1, State: Running, Role: LEADER
I20260619 12:04:55.911228 6006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:55.911293 6011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36637 } }
I20260619 12:04:55.911779 6012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36637 } } }
I20260619 12:04:55.911818 6013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebc6fe1ae8f48cb95d2fd34899f85f1. Latest consensus state: current_term: 1 leader_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebc6fe1ae8f48cb95d2fd34899f85f1" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36637 } } }
I20260619 12:04:55.911926 6013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:55.912180 6020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:55.912179 6012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:55.912803 6020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:55.914651 6020 catalog_manager.cc:1347] Generated new cluster ID: a0f4a40b6ac14e63b416c16560ab8739
I20260619 12:04:55.914716 6020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:55.917891 6020 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:55.918507 6020 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:55.924795 6020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ebc6fe1ae8f48cb95d2fd34899f85f1: Generated new TSK 0
I20260619 12:04:55.925016 6020 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 12:04:55.948956 6036 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 12:04:55.949066 6036 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260619 12:04:55.949553 6035 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:55.933644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:36637 (local address 127.0.0.1:37488)
0619 12:04:55.933886 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:55.933898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:55.933983 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:55.934163 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:55.934164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:55.934166 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:04:55.934238 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:55.934240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.934903 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.934904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:55.935569 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:55.935571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.935912 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.935913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:55.935974 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.935996 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:04:55.949208 (+ 13212us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 12:04:55.949300 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.203.190:36637: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":140,"threads_started":1}
I20260619 12:04:55.949966 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:55.933721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.0.203.190:36637)
0619 12:04:55.933776 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:55.933778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:55.933971 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:04:55.934083 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:55.934084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:55.934096 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:55.934151 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:04:55.934297 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.934817 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.935646 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:55.935850 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:55.935905 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:55.936102 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:04:55.949047 (+ 12945us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 12:04:55.949774 (+ 727us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 12:04:55.950044 6009 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.203.190:36637: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 12:04:55.950140 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 5939
2026-06-19T12:04:55Z chronyd exiting
I20260619 12:04:55.986162 814 test_util.cc:175] -----------------------------------------------
I20260619 12:04:55.986260 814 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870673623844-814-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 12:04:55.986990 814 mini_oidc.cc:152] Starting JWKS server
I20260619 12:04:55.987753 814 webserver.cc:460] Webserver started at https://127.0.0.1:39767/ using document root <none> and password file <none>
I20260619 12:04:55.987802 814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:04:55.987920 814 webserver.cc:460] Webserver started at http://127.0.0.1:33993/ using document root <none> and password file <none>
2026-06-19T12:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:55Z Disabled control of system clock
I20260619 12:04:55.997186 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:40989
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:37453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:40989
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39767/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/testchainca.cert with env {}
W20260619 12:04:56.083098 6042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:56.083323 6042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:56.083346 6042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:56.084609 6042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:56.084659 6042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:56.084671 6042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:56.084682 6042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:56.084693 6042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:04:56.084704 6042 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:04:56.084724 6042 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39767/jwks.json
I20260619 12:04:56.086263 6042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:40989
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39767/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:56.086495 6042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:56.086680 6042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:56.091938 6047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:56.091938 6048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:56.092054 6042 server_base.cc:1034] running on GCE node
W20260619 12:04:56.091976 6050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:56.092532 6042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:56.092846 6042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:56.093983 6042 hybrid_clock.cc:648] HybridClock initialized: now 1781870696093973 us; error 39 us; skew 500 ppm
I20260619 12:04:56.095508 6042 webserver.cc:460] Webserver started at http://127.0.203.190:40223/ using document root <none> and password file <none>
I20260619 12:04:56.095785 6042 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:56.095844 6042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:56.095955 6042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:56.097039 6042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "88fbd12918c34a21bd5fa9a30c4ecada"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "d29a5671e54741fa446698f05dea35a1"
server_key_iv: "b2646c3c880e832555a7233fab10b090"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.097374 6042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "88fbd12918c34a21bd5fa9a30c4ecada"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "d29a5671e54741fa446698f05dea35a1"
server_key_iv: "b2646c3c880e832555a7233fab10b090"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.098737 6042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:56.099660 6056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:56.099913 6042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:56.100025 6042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "88fbd12918c34a21bd5fa9a30c4ecada"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "d29a5671e54741fa446698f05dea35a1"
server_key_iv: "b2646c3c880e832555a7233fab10b090"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.100116 6042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:56.131186 6042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:56.131631 6042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:56.131762 6042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:56.136591 6042 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:40989
I20260619 12:04:56.136765 6108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:40989 every 8 connection(s)
I20260619 12:04:56.137063 6042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:56.138146 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:56.140758 6109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada: Bootstrap starting.
I20260619 12:04:56.141350 6109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:56.141625 6109 log.cc:824] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:56.142432 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6042
I20260619 12:04:56.142534 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:56.142568 6109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada: No bootstrap required, opened a new log
I20260619 12:04:56.142724 814 external_mini_cluster.cc:1428] Setting key f8b07c5bcf6d6bd06e4cb2da77c01f8b
I20260619 12:04:56.144533 6109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40989 } }
I20260619 12:04:56.144706 6109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:56.144742 6109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88fbd12918c34a21bd5fa9a30c4ecada, State: Initialized, Role: FOLLOWER
I20260619 12:04:56.144912 6109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40989 } }
I20260619 12:04:56.145006 6109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:56.145047 6109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:56.145107 6109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:56.145737 6109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40989 } }
I20260619 12:04:56.145831 6109 leader_election.cc:302] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88fbd12918c34a21bd5fa9a30c4ecada; no voters:
I20260619 12:04:56.146009 6109 leader_election.cc:288] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:56.146090 6114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:56.146281 6114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [term 1 LEADER]: Becoming Leader. State: Replica: 88fbd12918c34a21bd5fa9a30c4ecada, State: Running, Role: LEADER
I20260619 12:04:56.146298 6109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:56.146389 6114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40989 } }
I20260619 12:04:56.146840 6114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88fbd12918c34a21bd5fa9a30c4ecada. Latest consensus state: current_term: 1 leader_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40989 } } }
I20260619 12:04:56.146944 6114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:56.147135 6115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fbd12918c34a21bd5fa9a30c4ecada" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 40989 } } }
I20260619 12:04:56.147205 6115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:56.147269 6123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:56.147895 6123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:56.150074 6123 catalog_manager.cc:1347] Generated new cluster ID: b8e3d90aa9e54b43abe434c6ae548b5e
I20260619 12:04:56.150146 6123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:56.150588 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:56.143327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.0.203.190:40989)
0619 12:04:56.143570 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:56.143579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:56.143615 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:04:56.144490 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:56.144498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:56.144575 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:56.144738 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:56.145042 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.146722 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.147961 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.148673 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.148802 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.149649 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:56.149653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:56.149653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:56.149856 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:56.149912 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:56.149925 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:56.150290 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:56.150324 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 12:04:56.150399 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260619 12:04:56.152146 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:04:56.159497 6123 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:56.160207 6123 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:56.164862 6123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88fbd12918c34a21bd5fa9a30c4ecada: Generated new TSK 0
I20260619 12:04:56.165086 6123 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 12:04:56.191303 6139 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 12:04:56.191437 6139 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 12:04:56.191879 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:56.176400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34808 (local address 127.0.203.190:40989)
0619 12:04:56.176480 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:56.176483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:56.176596 (+ 113us) server_negotiation.cc:407] Connection header received
0619 12:04:56.176687 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:56.176688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:56.176704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:56.176817 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:04:56.176965 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.177762 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.178674 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.179001 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.179087 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.179349 (+ 262us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:04:56.191599 (+ 12250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 12:04:56.191783 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 12:04:56.191967 6113 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 12:04:56.191979 6138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:56.176294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:40989 (local address 127.0.0.1:34808)
0619 12:04:56.176510 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:56.176521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:56.176605 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:56.176821 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:56.176821 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:56.176824 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:04:56.176898 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:56.176901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.177889 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.177890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:56.178556 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:56.178557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.179130 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.179131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:56.179209 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.179217 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:04:56.191750 (+ 12533us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 12:04:56.191887 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.203.190:40989: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":151,"threads_started":1}
I20260619 12:04:56.192497 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6042
2026-06-19T12:04:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 12:04:56.231159 814 mini_oidc.cc:152] Starting JWKS server
I20260619 12:04:56.232254 814 webserver.cc:460] Webserver started at https://127.0.0.1:43201/ using document root <none> and password file <none>
I20260619 12:04:56.232338 814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:04:56.232488 814 webserver.cc:460] Webserver started at http://127.0.0.1:35135/ using document root <none> and password file <none>
2026-06-19T12:04:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:56Z Disabled control of system clock
I20260619 12:04:56.243018 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38231
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:36735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:38231
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43201/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 12:04:56.329382 6145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:56.329758 6145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:56.329805 6145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:56.331290 6145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:56.331354 6145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:56.331403 6145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:56.331415 6145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:56.331439 6145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:04:56.331451 6145 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:04:56.331462 6145 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43201/jwks.json
I20260619 12:04:56.333117 6145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:38231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43201/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:56.333375 6145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:56.333657 6145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:56.338572 6151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:56.338584 6153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:56.338680 6150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:56.339174 6145 server_base.cc:1034] running on GCE node
I20260619 12:04:56.339624 6145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:56.340133 6145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:56.341379 6145 hybrid_clock.cc:648] HybridClock initialized: now 1781870696341363 us; error 38 us; skew 500 ppm
I20260619 12:04:56.342972 6145 webserver.cc:460] Webserver started at http://127.0.203.190:38261/ using document root <none> and password file <none>
I20260619 12:04:56.343235 6145 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:56.343287 6145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:56.343446 6145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:56.344570 6145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "32aaa05a71c54f0581c01d397d22a6c7"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "ca5a240c9f79bef0361efe4e99b72915"
server_key_iv: "d0dd35bbb5a8f3d0f05d164165801750"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.344928 6145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "32aaa05a71c54f0581c01d397d22a6c7"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "ca5a240c9f79bef0361efe4e99b72915"
server_key_iv: "d0dd35bbb5a8f3d0f05d164165801750"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.346437 6145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:56.347484 6159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:56.347699 6145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260619 12:04:56.347810 6145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "32aaa05a71c54f0581c01d397d22a6c7"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "ca5a240c9f79bef0361efe4e99b72915"
server_key_iv: "d0dd35bbb5a8f3d0f05d164165801750"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.347890 6145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:56.362299 6145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:56.362643 6145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:56.362777 6145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:56.367290 6145 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:38231
I20260619 12:04:56.367503 6211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:38231 every 8 connection(s)
I20260619 12:04:56.367741 6145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:56.368593 6212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:56.368593 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6145
I20260619 12:04:56.368696 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:56.368884 814 external_mini_cluster.cc:1428] Setting key e0700e26b55394da1c34d464b39d033f
I20260619 12:04:56.373124 6212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7: Bootstrap starting.
I20260619 12:04:56.373950 6212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:56.374336 6212 log.cc:824] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:56.375449 6212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7: No bootstrap required, opened a new log
I20260619 12:04:56.375916 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:56.369515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.0.203.190:38231)
0619 12:04:56.369788 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:56.369799 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:56.369841 (+ 42us) server_negotiation.cc:407] Connection header received
0619 12:04:56.370473 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:56.370484 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:56.370583 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:56.370786 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:56.370973 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.372438 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.373344 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.373991 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.374153 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.375005 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:56.375011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:56.375012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:56.375251 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:56.375302 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:56.375320 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:56.375414 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:56.375508 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 12:04:56.375616 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260619 12:04:56.377287 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:04:56.377828 6212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32aaa05a71c54f0581c01d397d22a6c7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38231 } }
I20260619 12:04:56.378026 6212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:56.378114 6212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32aaa05a71c54f0581c01d397d22a6c7, State: Initialized, Role: FOLLOWER
I20260619 12:04:56.378275 6212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32aaa05a71c54f0581c01d397d22a6c7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38231 } }
I20260619 12:04:56.378363 6212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:56.378389 6212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:56.378444 6212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:56.379247 6212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32aaa05a71c54f0581c01d397d22a6c7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38231 } }
I20260619 12:04:56.379406 6212 leader_election.cc:302] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32aaa05a71c54f0581c01d397d22a6c7; no voters:
I20260619 12:04:56.379623 6212 leader_election.cc:288] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:56.379726 6217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:56.380002 6212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:56.379999 6217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [term 1 LEADER]: Becoming Leader. State: Replica: 32aaa05a71c54f0581c01d397d22a6c7, State: Running, Role: LEADER
I20260619 12:04:56.380280 6217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32aaa05a71c54f0581c01d397d22a6c7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38231 } }
I20260619 12:04:56.380877 6219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32aaa05a71c54f0581c01d397d22a6c7. Latest consensus state: current_term: 1 leader_uuid: "32aaa05a71c54f0581c01d397d22a6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32aaa05a71c54f0581c01d397d22a6c7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38231 } } }
I20260619 12:04:56.380995 6219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:56.381109 6218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32aaa05a71c54f0581c01d397d22a6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32aaa05a71c54f0581c01d397d22a6c7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38231 } } }
I20260619 12:04:56.381170 6218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:56.381274 6226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:56.381873 6226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:56.384624 6226 catalog_manager.cc:1347] Generated new cluster ID: 4dc5e4e1e9144a53a6134ddec50b8212
I20260619 12:04:56.384713 6226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:56.399619 6226 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:56.400537 6226 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:56.406755 6226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32aaa05a71c54f0581c01d397d22a6c7: Generated new TSK 0
I20260619 12:04:56.407015 6226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:04:56.418480 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:56.414356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.0.203.190:38231)
0619 12:04:56.414453 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:56.414456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:56.414619 (+ 163us) server_negotiation.cc:407] Connection header received
0619 12:04:56.414679 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:56.414680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:56.414695 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:56.414778 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:04:56.414959 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.415560 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.416809 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.417006 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.417077 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.417277 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:04:56.418258 (+ 981us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 12:04:56.418399 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36472: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 12:04:56.418572 6215 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36472: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 12:04:56.418648 6241 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:56.414268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38231 (local address 127.0.0.1:36472)
0619 12:04:56.414476 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:56.414492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:56.414609 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:56.414799 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:56.414800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:56.414803 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:04:56.414889 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:56.414892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.415643 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.415644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:56.416687 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:56.416690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.417103 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.417104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:56.417160 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.417167 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:04:56.418372 (+ 1205us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 12:04:56.418509 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.203.190:38231: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
I20260619 12:04:56.419027 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6145
E20260619 12:04:56.430120 6242 webserver.cc:533] Webserver: sslize failed:
E20260619 12:04:56.430220 6242 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T12:04:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T12:04:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:56Z Disabled control of system clock
I20260619 12:04:56.465186 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:44095
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:45743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:44095
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/testchainca.cert with env {}
W20260619 12:04:56.547317 6246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:56.547590 6246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:56.547613 6246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:56.548874 6246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:56.548923 6246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:56.548935 6246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:56.548946 6246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:56.548959 6246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:04:56.548969 6246 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:04:56.548979 6246 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 12:04:56.550599 6246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:44095
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:44095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:56.550840 6246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:56.551011 6246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:56.556269 6252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:56.556300 6251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:56.556263 6254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:56.556310 6246 server_base.cc:1034] running on GCE node
I20260619 12:04:56.556866 6246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:56.557206 6246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:56.558377 6246 hybrid_clock.cc:648] HybridClock initialized: now 1781870696558364 us; error 32 us; skew 500 ppm
I20260619 12:04:56.559937 6246 webserver.cc:460] Webserver started at http://127.0.203.190:33321/ using document root <none> and password file <none>
I20260619 12:04:56.560197 6246 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:56.560247 6246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:56.560375 6246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:56.561534 6246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "c1ef7334a2974b32ad838fe06386f9ab"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "e46b17f7248e976da6f436ebfaaf9f1f"
server_key_iv: "83bc95aff3d6d9689e9af180267b0ce6"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.561914 6246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "c1ef7334a2974b32ad838fe06386f9ab"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "e46b17f7248e976da6f436ebfaaf9f1f"
server_key_iv: "83bc95aff3d6d9689e9af180267b0ce6"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.563560 6246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:56.564509 6260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:56.564787 6246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:56.564896 6246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "c1ef7334a2974b32ad838fe06386f9ab"
format_stamp: "Formatted at 2026-06-19 12:04:56 on dist-test-slave-44vj"
server_key: "e46b17f7248e976da6f436ebfaaf9f1f"
server_key_iv: "83bc95aff3d6d9689e9af180267b0ce6"
server_key_version: "encryptionkey@0"
I20260619 12:04:56.564971 6246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:56.584651 6246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:56.584972 6246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:56.585103 6246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:56.589293 6246 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:44095
I20260619 12:04:56.589411 6312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:44095 every 8 connection(s)
I20260619 12:04:56.589722 6246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:56.590370 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6246
I20260619 12:04:56.590458 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:56.590634 814 external_mini_cluster.cc:1428] Setting key ce413ddd0ea4bd478cde1cc1d085b535
I20260619 12:04:56.590935 6313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:56.594631 6313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab: Bootstrap starting.
I20260619 12:04:56.595465 6313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:56.595809 6313 log.cc:824] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:56.596587 6313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab: No bootstrap required, opened a new log
I20260619 12:04:56.596797 6316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:56.591149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39984 (local address 127.0.203.190:44095)
0619 12:04:56.591395 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:56.591405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:56.591445 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:04:56.591904 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:56.591911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:56.591982 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:56.592123 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:56.592381 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.593302 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.594399 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.594864 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.594968 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.595931 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:56.595937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:56.595938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:04:56.596169 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:56.596209 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:04:56.596222 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:56.596296 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:56.596416 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 12:04:56.596513 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260619 12:04:56.597882 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:04:56.598806 6313 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ef7334a2974b32ad838fe06386f9ab" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 44095 } }
I20260619 12:04:56.599010 6313 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:56.599038 6313 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ef7334a2974b32ad838fe06386f9ab, State: Initialized, Role: FOLLOWER
I20260619 12:04:56.599160 6313 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ef7334a2974b32ad838fe06386f9ab" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 44095 } }
I20260619 12:04:56.599239 6313 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:56.599262 6313 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:56.599311 6313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:56.600071 6313 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ef7334a2974b32ad838fe06386f9ab" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 44095 } }
I20260619 12:04:56.600184 6313 leader_election.cc:302] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1ef7334a2974b32ad838fe06386f9ab; no voters:
I20260619 12:04:56.600374 6313 leader_election.cc:288] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:56.600579 6319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:56.600684 6313 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:56.600725 6319 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [term 1 LEADER]: Becoming Leader. State: Replica: c1ef7334a2974b32ad838fe06386f9ab, State: Running, Role: LEADER
I20260619 12:04:56.600875 6319 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ef7334a2974b32ad838fe06386f9ab" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 44095 } }
I20260619 12:04:56.601414 6319 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1ef7334a2974b32ad838fe06386f9ab. Latest consensus state: current_term: 1 leader_uuid: "c1ef7334a2974b32ad838fe06386f9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ef7334a2974b32ad838fe06386f9ab" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 44095 } } }
I20260619 12:04:56.601501 6319 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:56.601444 6318 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1ef7334a2974b32ad838fe06386f9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ef7334a2974b32ad838fe06386f9ab" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 44095 } } }
I20260619 12:04:56.601570 6318 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab [sys.catalog]: This master's current role is: LEADER
W20260619 12:04:56.602793 6333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:04:56.602843 6333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:04:56.602923 6327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:56.603133 6327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:56.604893 6327 catalog_manager.cc:1347] Generated new cluster ID: 709c12cde84d4aac8be59db8146395bd
I20260619 12:04:56.604959 6327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:56.644721 6327 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:56.645293 6327 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:56.656378 6327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1ef7334a2974b32ad838fe06386f9ab: Generated new TSK 0
I20260619 12:04:56.656607 6327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:04:56.762230 6316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:56.667006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.0.203.190:44095)
0619 12:04:56.667070 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:56.667072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:56.667106 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:04:56.667190 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:56.667191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:56.667209 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:56.667299 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:04:56.667449 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.667984 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.668829 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.669040 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.669093 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.669276 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:04:56.761980 (+ 92704us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 12:04:56.762163 (+ 183us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39996: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 12:04:56.762305 6316 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39996: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 12:04:56.762482 6342 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:56.666904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:44095 (local address 127.0.0.1:39996)
0619 12:04:56.667051 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:56.667061 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:56.667125 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:56.667300 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:56.667301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:56.667303 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:04:56.667398 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:56.667400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.668078 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.668079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:56.668739 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:56.668741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:56.669111 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:56.669112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:56.669174 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:56.669180 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:04:56.762162 (+ 92982us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 12:04:56.762345 (+ 183us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.203.190:44095: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260619 12:04:56.762895 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6246
2026-06-19T12:04:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (323 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:04:56 dist-test-slave-44vj krb5kdc[6354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:04:56 dist-test-slave-44vj krb5kdc[6354](info): set up 2 sockets
Jun 19 12:04:56 dist-test-slave-44vj krb5kdc[6354](info): commencing operation
krb5kdc: starting...
W20260619 12:04:57.915695 814 mini_kdc.cc:121] Time spent starting KDC: real 0.958s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:04:57 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:04:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:04:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:04:58.037921 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36863
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:36863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:58.119781 6367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:58.120012 6367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:58.120035 6367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:58.121269 6367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:04:58.121326 6367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:58.121340 6367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:58.121351 6367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:04:58.121362 6367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:04:58.122951 6367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:36863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:58.123208 6367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:58.123481 6367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:04:58.128827 6367 server_base.cc:1034] running on GCE node
W20260619 12:04:58.129971 6373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.130399 6375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.130669 6372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:58.130978 6367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:58.131388 6367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:58.132591 6367 hybrid_clock.cc:648] HybridClock initialized: now 1781870698132564 us; error 47 us; skew 500 ppm
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:58.135453 6367 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:04:58.136054 6367 webserver.cc:460] Webserver started at http://127.0.203.190:45053/ using document root <none> and password file <none>
I20260619 12:04:58.136291 6367 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:58.136343 6367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:58.136466 6367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:58.137517 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "adaf66386b91458bae4ce00aaddf4496"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "2dec226fec8126213f4f6551d7268f4c"
server_key_iv: "e60a1f0a4c637f93f33c08bfd254ab75"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.137866 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "adaf66386b91458bae4ce00aaddf4496"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "2dec226fec8126213f4f6551d7268f4c"
server_key_iv: "e60a1f0a4c637f93f33c08bfd254ab75"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.139220 6367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:58.140126 6382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.140347 6367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:58.140431 6367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "adaf66386b91458bae4ce00aaddf4496"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "2dec226fec8126213f4f6551d7268f4c"
server_key_iv: "e60a1f0a4c637f93f33c08bfd254ab75"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.140504 6367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:58.152679 6367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:58.155483 6367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:58.155653 6367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:58.160687 6367 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:36863
I20260619 12:04:58.160768 6434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:36863 every 8 connection(s)
I20260619 12:04:58.161298 6367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:04:58.162037 6435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:04:58.163475 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6367
I20260619 12:04:58.163585 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:04:58.163779 814 external_mini_cluster.cc:1428] Setting key 07c60845c6ab0c0b15654f7bfd0ca566
I20260619 12:04:58.164827 6435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496: Bootstrap starting.
I20260619 12:04:58.165488 6435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496: Neither blocks nor log segments found. Creating new log.
I20260619 12:04:58.165853 6435 log.cc:824] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496: Log is configured to *not* fsync() on all Append() calls
I20260619 12:04:58.166721 6435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496: No bootstrap required, opened a new log
I20260619 12:04:58.168874 6435 raft_consensus.cc:348] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adaf66386b91458bae4ce00aaddf4496" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36863 } }
I20260619 12:04:58.169080 6435 raft_consensus.cc:374] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:04:58.169119 6435 raft_consensus.cc:729] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adaf66386b91458bae4ce00aaddf4496, State: Initialized, Role: FOLLOWER
I20260619 12:04:58.169266 6435 consensus_queue.cc:260] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adaf66386b91458bae4ce00aaddf4496" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36863 } }
I20260619 12:04:58.169351 6435 raft_consensus.cc:388] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:04:58.169376 6435 raft_consensus.cc:482] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:04:58.169410 6435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:04:58.170099 6435 raft_consensus.cc:504] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adaf66386b91458bae4ce00aaddf4496" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36863 } }
I20260619 12:04:58.170208 6435 leader_election.cc:302] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adaf66386b91458bae4ce00aaddf4496; no voters:
I20260619 12:04:58.170400 6435 leader_election.cc:288] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:04:58.170501 6440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:04:58.170686 6435 sys_catalog.cc:564] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:04:58.170775 6440 raft_consensus.cc:686] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [term 1 LEADER]: Becoming Leader. State: Replica: adaf66386b91458bae4ce00aaddf4496, State: Running, Role: LEADER
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:58.170917 6440 consensus_queue.cc:237] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adaf66386b91458bae4ce00aaddf4496" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36863 } }
I20260619 12:04:58.171435 6441 sys_catalog.cc:455] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adaf66386b91458bae4ce00aaddf4496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adaf66386b91458bae4ce00aaddf4496" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36863 } } }
I20260619 12:04:58.171535 6441 sys_catalog.cc:458] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:58.171823 6440 sys_catalog.cc:455] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adaf66386b91458bae4ce00aaddf4496. Latest consensus state: current_term: 1 leader_uuid: "adaf66386b91458bae4ce00aaddf4496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adaf66386b91458bae4ce00aaddf4496" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36863 } } }
I20260619 12:04:58.171923 6449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:04:58.171913 6440 sys_catalog.cc:458] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496 [sys.catalog]: This master's current role is: LEADER
I20260619 12:04:58.172514 6449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:04:58.174628 6449 catalog_manager.cc:1347] Generated new cluster ID: 8c9f55ed17ef472e8d493811b68cfd7b
I20260619 12:04:58.174690 6449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:04:58.175762 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:58.164407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.0.203.190:36863)
0619 12:04:58.164597 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:58.164607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:58.164640 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:04:58.165226 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:58.165234 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:58.165306 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:58.165451 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:58.166259 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.167247 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.168015 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.168441 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.168539 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:58.171494 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:58.171500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:58.171500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:58.171517 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:58.173907 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.174396 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.174399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.174400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.174522 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.174777 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.174778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.174778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.175039 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:58.175176 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:58.175312 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 12:04:58.175505 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260619 12:04:58.188005 6449 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:04:58.188722 6449 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:04:58.195533 6449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adaf66386b91458bae4ce00aaddf4496: Generated new TSK 0
I20260619 12:04:58.195768 6449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:04:58.228551 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36863
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:58.317816 6461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:58.318081 6461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:58.318116 6461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:58.319489 6461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:58.319553 6461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:58.319603 6461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:04:58.321374 6461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:58.321669 6461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:58.321918 6461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:58.327636 6469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.327636 6466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.327658 6467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:58.327884 6461 server_base.cc:1034] running on GCE node
I20260619 12:04:58.328146 6461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:58.328487 6461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:58.329655 6461 hybrid_clock.cc:648] HybridClock initialized: now 1781870698329634 us; error 46 us; skew 500 ppm
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:58.332489 6461 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:04:58.332970 6461 webserver.cc:460] Webserver started at http://127.0.203.129:40931/ using document root <none> and password file <none>
I20260619 12:04:58.333201 6461 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:58.333248 6461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:58.333352 6461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:58.334398 6461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "265bc2f983af482faf865ed79b923a84"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "e952673b0f3fb5eff0c009a7dae45e53"
server_key_iv: "730a0fdd79be0a76bef40140f0c86f54"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.334717 6461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "265bc2f983af482faf865ed79b923a84"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "e952673b0f3fb5eff0c009a7dae45e53"
server_key_iv: "730a0fdd79be0a76bef40140f0c86f54"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.336148 6461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:58.337060 6476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.337272 6461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:58.337360 6461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "265bc2f983af482faf865ed79b923a84"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "e952673b0f3fb5eff0c009a7dae45e53"
server_key_iv: "730a0fdd79be0a76bef40140f0c86f54"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.337441 6461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:58.353914 6461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:58.356343 6461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:58.356478 6461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:58.356739 6461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:58.357065 6461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:58.357103 6461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.357126 6461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:58.357177 6461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.363564 6461 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:40271
I20260619 12:04:58.363711 6589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:40271 every 8 connection(s)
I20260619 12:04:58.363977 6461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:04:58.364794 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6461
I20260619 12:04:58.364908 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:04:58.365101 814 external_mini_cluster.cc:1428] Setting key c3784d1125159fc5daea238df0ce7479
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
I20260619 12:04:58.375064 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:58.364887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:55461 (local address 127.0.203.190:36863)
0619 12:04:58.364959 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:58.364961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:58.365219 (+ 258us) server_negotiation.cc:407] Connection header received
0619 12:04:58.366310 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:58.366311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:58.366324 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:58.366400 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:58.367042 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.367799 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.368939 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.369155 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.369214 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:58.372057 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:58.372058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:58.372059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:58.372067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:58.373511 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.373935 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.373937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.373937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.373992 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.374221 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.374222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.374223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.374359 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:58.374427 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:58.374886 (+ 459us) server_negotiation.cc:299] Negotiation successful
0619 12:04:58.374983 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:04:58.375174 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:58.364822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:36863 (local address 127.0.203.129:55461)
0619 12:04:58.365081 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:58.365116 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:58.366183 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:58.366458 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:58.366462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:58.366745 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:58.366949 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:58.366953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.367911 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.367913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:58.368844 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:58.368848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.369228 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.369229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:58.369371 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:58.369951 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:58.369962 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:58.371913 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:58.373588 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:58.373598 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:58.373601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:58.373868 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:58.374047 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:58.374048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:58.374048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:58.374166 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:58.374452 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:58.374463 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:58.374661 (+ 198us) client_negotiation.cc:769] Sending connection context
0619 12:04:58.374789 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 12:04:58.374960 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
I20260619 12:04:58.375797 6590 heartbeater.cc:347] Connected to a master server at 127.0.203.190:36863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
I20260619 12:04:58.375998 6590 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:58.376242 6590 heartbeater.cc:511] Master 127.0.203.190:36863 requested a full tablet report, sending...
I20260619 12:04:58.376940 6399 ts_manager.cc:194] Registered new tserver with Master: 265bc2f983af482faf865ed79b923a84 (127.0.203.129:40271)
I20260619 12:04:58.377718 6399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:55461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:04:58.390480 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36863
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:58.477310 6595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:58.477542 6595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:58.477567 6595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:58.478883 6595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:58.478946 6595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:58.478987 6595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:04:58.480684 6595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:58.480964 6595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:58.481235 6595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:58.486688 6603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.486826 6600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.486830 6601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:58.487109 6595 server_base.cc:1034] running on GCE node
I20260619 12:04:58.487389 6595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:58.487677 6595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:58.488847 6595 hybrid_clock.cc:648] HybridClock initialized: now 1781870698488835 us; error 29 us; skew 500 ppm
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:58.491925 6595 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:04:58.492448 6595 webserver.cc:460] Webserver started at http://127.0.203.130:38313/ using document root <none> and password file <none>
I20260619 12:04:58.492679 6595 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:58.492730 6595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:58.492869 6595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:58.494097 6595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "cd92b5a2af3a4059b878e956642727f3"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "eb8627c108a04a04d788e98f72ced07d"
server_key_iv: "73d9bcb76933f7f3e0536aeacd85f7bc"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.494474 6595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "cd92b5a2af3a4059b878e956642727f3"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "eb8627c108a04a04d788e98f72ced07d"
server_key_iv: "73d9bcb76933f7f3e0536aeacd85f7bc"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.496122 6595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:04:58.497052 6610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.497284 6595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:04:58.497380 6595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "cd92b5a2af3a4059b878e956642727f3"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "eb8627c108a04a04d788e98f72ced07d"
server_key_iv: "73d9bcb76933f7f3e0536aeacd85f7bc"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.497452 6595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:58.530441 6595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:58.532899 6595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:58.533063 6595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:58.533547 6595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:58.533959 6595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:58.533995 6595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.534029 6595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:58.534045 6595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.541679 6595 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:41185
I20260619 12:04:58.541788 6723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:41185 every 8 connection(s)
I20260619 12:04:58.542218 6595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:04:58.546406 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6595
I20260619 12:04:58.546509 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:04:58.546682 814 external_mini_cluster.cc:1428] Setting key c1ac0deb228a602efda2c3a558e4fa57
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:58.553401 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:58.543129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:36213 (local address 127.0.203.190:36863)
0619 12:04:58.543218 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:58.543221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:58.543425 (+ 204us) server_negotiation.cc:407] Connection header received
0619 12:04:58.544478 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:58.544480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:58.544495 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:58.544554 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:58.545219 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.545734 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.546763 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.546979 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.547050 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:58.550054 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:58.550055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:58.550055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:58.550065 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:58.551520 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.552178 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.552179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.552179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.552261 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.552624 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.552625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.552625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.552769 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:58.552841 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:58.553236 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 12:04:58.553325 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:04:58.553687 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:58.543077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:36863 (local address 127.0.203.130:36213)
0619 12:04:58.543258 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:58.543304 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:58.544373 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:58.544623 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:58.544628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:58.544915 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:58.545129 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:58.545133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.545814 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.545815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:58.546691 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:58.546694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.547094 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.547096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:58.547301 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:58.547938 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:58.547949 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:58.549897 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:58.551636 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:58.551650 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:58.551654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:58.552055 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:58.552354 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:58.552354 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:58.552355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:58.552521 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:58.552853 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:58.552860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:58.553066 (+ 206us) client_negotiation.cc:769] Sending connection context
0619 12:04:58.553210 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 12:04:58.553409 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260619 12:04:58.554379 6724 heartbeater.cc:347] Connected to a master server at 127.0.203.190:36863
I20260619 12:04:58.554499 6724 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:58.554733 6724 heartbeater.cc:511] Master 127.0.203.190:36863 requested a full tablet report, sending...
I20260619 12:04:58.555181 6399 ts_manager.cc:194] Registered new tserver with Master: cd92b5a2af3a4059b878e956642727f3 (127.0.203.130:41185)
I20260619 12:04:58.555790 6399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:36213
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:04:58.576272 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36863
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:04:58.663003 6729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:04:58.663240 6729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:04:58.663275 6729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:04:58.664566 6729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:04:58.664621 6729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:04:58.664660 6729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:04:58.666276 6729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:04:58.666543 6729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:04:58.666775 6729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:04:58.672509 6734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.672515 6737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:04:58.672510 6735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:04:58.672884 6729 server_base.cc:1034] running on GCE node
I20260619 12:04:58.673146 6729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:04:58.673413 6729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:04:58.674582 6729 hybrid_clock.cc:648] HybridClock initialized: now 1781870698674557 us; error 45 us; skew 500 ppm
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:58.678243 6729 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:04:58.678923 6729 webserver.cc:460] Webserver started at http://127.0.203.131:41755/ using document root <none> and password file <none>
I20260619 12:04:58.679198 6729 fs_manager.cc:359] Metadata directory not provided
I20260619 12:04:58.679332 6729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:04:58.679564 6729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:04:58.680832 6729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "194cd3ee723b46b0a973a325c3de5f7f"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "047a46842f88ac95ec67740e367957b3"
server_key_iv: "dc386cc7f08fae209639053604d35212"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.681197 6729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "194cd3ee723b46b0a973a325c3de5f7f"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "047a46842f88ac95ec67740e367957b3"
server_key_iv: "dc386cc7f08fae209639053604d35212"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.682571 6729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:04:58.683435 6744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.683645 6729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:04:58.683732 6729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "194cd3ee723b46b0a973a325c3de5f7f"
format_stamp: "Formatted at 2026-06-19 12:04:58 on dist-test-slave-44vj"
server_key: "047a46842f88ac95ec67740e367957b3"
server_key_iv: "dc386cc7f08fae209639053604d35212"
server_key_version: "encryptionkey@0"
I20260619 12:04:58.683807 6729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:04:58.713153 6729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:04:58.715453 6729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:04:58.715597 6729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:04:58.715838 6729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:04:58.716168 6729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:04:58.716225 6729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.716257 6729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:04:58.716274 6729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:04:58.722744 6729 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:37845
I20260619 12:04:58.722855 6857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:37845 every 8 connection(s)
I20260619 12:04:58.723173 6729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:04:58 dist-test-slave-44vj krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:04:58.732575 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6729
I20260619 12:04:58.732667 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:04:58.732851 814 external_mini_cluster.cc:1428] Setting key 2e506cae05a286bfc64d5e241c537d99
I20260619 12:04:58.734566 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:58.724123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:56763 (local address 127.0.203.190:36863)
0619 12:04:58.724210 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:58.724214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:58.724466 (+ 252us) server_negotiation.cc:407] Connection header received
0619 12:04:58.725606 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:58.725607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:58.725621 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:58.725676 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:58.726557 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.727124 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.728359 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.728562 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.728620 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:58.731143 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:58.731145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:58.731145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:58.731153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:58.732603 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.733320 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.733321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.733322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.733389 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:58.733730 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:58.733731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:58.733731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:58.733891 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:58.733967 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:58.734403 (+ 436us) server_negotiation.cc:299] Negotiation successful
0619 12:04:58.734488 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:04:58.734760 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:58.724066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:36863 (local address 127.0.203.131:56763)
0619 12:04:58.724265 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 12:04:58.724347 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0619 12:04:58.725467 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:04:58.725773 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:04:58.725777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:04:58.726108 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:04:58.726432 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:58.726439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.727220 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.727221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:58.728236 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:04:58.728264 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:04:58.728631 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:04:58.728632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:04:58.728779 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:58.729395 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:04:58.729405 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:04:58.731048 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:04:58.732748 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:58.732764 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:58.732768 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:58.733187 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:58.733463 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:04:58.733464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:04:58.733465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:04:58.733656 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:04:58.733996 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:04:58.734005 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:04:58.734216 (+ 211us) client_negotiation.cc:769] Sending connection context
0619 12:04:58.734349 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 12:04:58.734528 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260619 12:04:58.735280 6858 heartbeater.cc:347] Connected to a master server at 127.0.203.190:36863
I20260619 12:04:58.735430 6858 heartbeater.cc:464] Registering TS with master...
I20260619 12:04:58.735651 6858 heartbeater.cc:511] Master 127.0.203.190:36863 requested a full tablet report, sending...
I20260619 12:04:58.736116 6399 ts_manager.cc:194] Registered new tserver with Master: 194cd3ee723b46b0a973a325c3de5f7f (127.0.203.131:37845)
I20260619 12:04:58.736539 6399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:56763
I20260619 12:04:58.745613 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:04:59.378995 6590 heartbeater.cc:503] Master 127.0.203.190:36863 was elected leader, sending a full tablet report...
I20260619 12:04:59.556912 6724 heartbeater.cc:503] Master 127.0.203.190:36863 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/test-admin.keytab.
W20260619 12:04:59.704972 814 mini_kdc.cc:302] Time spent creating keytab for test-admin: real 0.959s user 0.001s sys 0.001s
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:59.707087 814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 12:04:59.709031 6865 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:59.708110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.0.203.129:40271)
0619 12:04:59.708295 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:59.708301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:59.708315 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:04:59.708346 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:59.708348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:59.708365 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:59.708444 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:59.708947 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33910: BlockingRecv error: recv got EOF from 127.0.0.1:33910 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":67,"threads_started":1}
W20260619 12:04:59.709152 814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:59.710326 814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:59.711206 814 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:59.725111 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:59.711847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.0.203.129:40271)
0619 12:04:59.711897 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:59.711899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:59.711966 (+ 67us) server_negotiation.cc:407] Connection header received
0619 12:04:59.712028 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:59.712029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:59.712041 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:59.712093 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:59.712493 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:59.713293 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:59.714119 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:59.714559 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:59.714654 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:59.721796 (+ 7142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:59.721814 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:59.721815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:59.721826 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:59.723852 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:59.724240 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:59.724244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:59.724245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:59.724301 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:59.724468 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:59.724469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:59.724469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:59.724633 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:59.724856 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:59.724903 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 12:04:59.724985 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 12:04:59.727469 6865 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:59.726779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.0.203.129:40271)
0619 12:04:59.726859 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:59.726860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:59.726923 (+ 63us) server_negotiation.cc:407] Connection header received
0619 12:04:59.726982 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:59.726982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:59.727000 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:59.727045 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:59.727429 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33922: BlockingRecv error: recv got EOF from 127.0.0.1:33922 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 12:04:59.727538 814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:59.728816 814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:59.729666 814 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:59.737402 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:59.730248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.0.203.129:40271)
0619 12:04:59.730318 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:59.730320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:59.730375 (+ 55us) server_negotiation.cc:407] Connection header received
0619 12:04:59.730437 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:59.730438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:59.730447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:59.730487 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:59.730849 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:59.731429 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:59.732148 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:59.732396 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:59.732470 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:59.734736 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:59.734738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:59.734739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:59.734747 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:59.736308 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:59.736672 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:59.736673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:59.736674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:59.736726 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:59.736890 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:59.736891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:59.736891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:59.737001 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:59.737107 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:59.737203 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 12:04:59.737289 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:04:59.737586 6858 heartbeater.cc:503] Master 127.0.203.190:36863 was elected leader, sending a full tablet report...
W20260619 12:04:59.739609 6865 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:04:59.738907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.0.203.129:40271)
0619 12:04:59.738969 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:59.738971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:59.738983 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:04:59.739074 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:59.739075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:59.739085 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:59.739135 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:59.739537 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33952: BlockingRecv error: recv got EOF from 127.0.0.1:33952 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 12:04:59.739761 814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870673623844-814-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:59.740963 814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:04:59.742025 814 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:04:59 dist-test-slave-44vj krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.129@KRBTEST.COM
I20260619 12:04:59.750439 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:04:59.742715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.0.203.129:40271)
0619 12:04:59.742787 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:04:59.742790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:04:59.743093 (+ 303us) server_negotiation.cc:407] Connection header received
0619 12:04:59.743158 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:04:59.743159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:04:59.743171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:04:59.743262 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:04:59.743757 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:59.744294 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:59.745157 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:04:59.745369 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:04:59.745450 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:04:59.747942 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:04:59.747944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:04:59.747944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:04:59.747954 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:04:59.749318 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:59.749690 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:59.749691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:59.749692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:59.749759 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:04:59.749956 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:04:59.749957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:04:59.749958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:04:59.750081 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:04:59.750175 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:04:59.750299 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 12:04:59.750364 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:04:59.751503 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6461
I20260619 12:04:59.756477 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6595
I20260619 12:04:59.760859 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6729
I20260619 12:04:59.765736 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6367
2026-06-19T12:04:59Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2828 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:05:00 dist-test-slave-44vj krb5kdc[6878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:05:00 dist-test-slave-44vj krb5kdc[6878](info): set up 2 sockets
Jun 19 12:05:00 dist-test-slave-44vj krb5kdc[6878](info): commencing operation
krb5kdc: starting...
W20260619 12:05:01.467839 814 mini_kdc.cc:121] Time spent starting KDC: real 1.682s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:05:01 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:05:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab.
Entry for principal oryx/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab.
I20260619 12:05:01.616375 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:35585
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab
--principal=oryx/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:35585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:01.717818 6891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:01.718156 6891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:01.718196 6891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:01.719805 6891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:01.719878 6891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:01.719908 6891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:01.719925 6891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:01.719941 6891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:01.722011 6891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:35585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab
--principal=oryx/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:35585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:01.722465 6891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:01.722752 6891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:01.729117 6896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:01.729286 6897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:01.729447 6891 server_base.cc:1034] running on GCE node
W20260619 12:05:01.729188 6899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:01.729911 6891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:01.730335 6891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:01.731493 6891 hybrid_clock.cc:648] HybridClock initialized: now 1781870701731476 us; error 42 us; skew 500 ppm
Jun 19 12:05:01 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:01.735039 6891 init.cc:377] Logged in from keytab as oryx/127.0.203.190@KRBTEST.COM (short username oryx)
I20260619 12:05:01.735636 6891 webserver.cc:460] Webserver started at http://127.0.203.190:39317/ using document root <none> and password file <none>
I20260619 12:05:01.735908 6891 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:01.735967 6891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:01.736091 6891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:01.737347 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa"
format_stamp: "Formatted at 2026-06-19 12:05:01 on dist-test-slave-44vj"
server_key: "14d776c85c08955c756a5e74b7fa48b9"
server_key_iv: "c1c3cd6d13f83eb2a6faa31778de282d"
server_key_version: "encryptionkey@0"
I20260619 12:05:01.737706 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa"
format_stamp: "Formatted at 2026-06-19 12:05:01 on dist-test-slave-44vj"
server_key: "14d776c85c08955c756a5e74b7fa48b9"
server_key_iv: "c1c3cd6d13f83eb2a6faa31778de282d"
server_key_version: "encryptionkey@0"
I20260619 12:05:01.739401 6891 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:05:01.740484 6906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:01.740887 6891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:01.741015 6891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa"
format_stamp: "Formatted at 2026-06-19 12:05:01 on dist-test-slave-44vj"
server_key: "14d776c85c08955c756a5e74b7fa48b9"
server_key_iv: "c1c3cd6d13f83eb2a6faa31778de282d"
server_key_version: "encryptionkey@0"
I20260619 12:05:01.741114 6891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:01.755175 6891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:01.757726 6891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:01.757854 6891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:01.762543 6891 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:35585
I20260619 12:05:01.762610 6958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:35585 every 8 connection(s)
I20260619 12:05:01.763072 6891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:05:01.763630 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6891
I20260619 12:05:01.763784 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:05:01.763978 814 external_mini_cluster.cc:1428] Setting key 3efd5ce27622bf765f40745e9dd06293
I20260619 12:05:01.764863 6959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:01.768384 6959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Bootstrap starting.
I20260619 12:05:01.769169 6959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:01.769721 6959 log.cc:824] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:01.770854 6959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: No bootstrap required, opened a new log
Jun 19 12:05:01 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:01.773833 6959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } }
I20260619 12:05:01.774052 6959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:01.774088 6959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5843ca67bb6f4ea1ad6ddc7965a21eaa, State: Initialized, Role: FOLLOWER
I20260619 12:05:01.774353 6959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } }
I20260619 12:05:01.774449 6959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:05:01.774487 6959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:05:01.774554 6959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:01.775557 6959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } }
I20260619 12:05:01.775702 6959 leader_election.cc:302] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5843ca67bb6f4ea1ad6ddc7965a21eaa; no voters:
I20260619 12:05:01.776131 6959 leader_election.cc:288] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:05:01.776197 6964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:01.776371 6964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 1 LEADER]: Becoming Leader. State: Replica: 5843ca67bb6f4ea1ad6ddc7965a21eaa, State: Running, Role: LEADER
I20260619 12:05:01.776543 6964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } }
I20260619 12:05:01.776839 6959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:01.777070 6964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5843ca67bb6f4ea1ad6ddc7965a21eaa. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } }
I20260619 12:05:01.777189 6964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:01.777335 6964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } }
I20260619 12:05:01.777397 6964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:01.778155 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:01.764639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.0.203.190:35585)
0619 12:05:01.764841 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:01.764852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:01.764927 (+ 75us) server_negotiation.cc:407] Connection header received
0619 12:05:01.765614 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:01.765627 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:01.765702 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:01.765922 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:01.766944 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:01.768023 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:01.768845 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:01.769364 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:01.769481 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:01.772858 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:01.772866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:01.772867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:01.772887 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:01.775240 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:01.775978 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:01.775982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:01.775983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:01.776143 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:01.776451 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:01.776451 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:01.776452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:01.776867 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:01.777099 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:01.777706 (+ 607us) server_negotiation.cc:299] Negotiation successful
0619 12:05:01.777892 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
W20260619 12:05:01.779248 6980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:05:01.779299 6980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:05:01.779378 6968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:01.779563 6968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:01.781332 6968 catalog_manager.cc:1347] Generated new cluster ID: d887bc3a8d6d40df98ced7921a80c4f8
I20260619 12:05:01.781394 6968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:05:01.785066 6968 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:05:01.785655 6968 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:01.797091 6968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Generated new TSK 0
I20260619 12:05:01.797345 6968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab.
Entry for principal oryx/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab.
I20260619 12:05:01.831010 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35585
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab
--principal=oryx/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:01.922653 6985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:01.922917 6985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:01.922942 6985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:01.924185 6985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:01.924227 6985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:01.924253 6985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:01.926008 6985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab
--principal=oryx/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:01.926277 6985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:01.926514 6985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:01.932488 6993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:01.932534 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:01.932585 6985 server_base.cc:1034] running on GCE node
W20260619 12:05:01.932534 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:01.933022 6985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:01.933324 6985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:01.934499 6985 hybrid_clock.cc:648] HybridClock initialized: now 1781870701934481 us; error 44 us; skew 500 ppm
Jun 19 12:05:01 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:01.937793 6985 init.cc:377] Logged in from keytab as oryx/127.0.203.129@KRBTEST.COM (short username oryx)
I20260619 12:05:01.938414 6985 webserver.cc:460] Webserver started at http://127.0.203.129:46879/ using document root <none> and password file <none>
I20260619 12:05:01.938656 6985 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:01.938707 6985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:01.938848 6985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:01.940253 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "0dfb4466a7ac441baa3bfd343cbf8704"
format_stamp: "Formatted at 2026-06-19 12:05:01 on dist-test-slave-44vj"
server_key: "af80f2bac5e4d68fd5db0d0dba23d9e4"
server_key_iv: "affc21d662c2c384e365a45e0631c35f"
server_key_version: "encryptionkey@0"
I20260619 12:05:01.940666 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "0dfb4466a7ac441baa3bfd343cbf8704"
format_stamp: "Formatted at 2026-06-19 12:05:01 on dist-test-slave-44vj"
server_key: "af80f2bac5e4d68fd5db0d0dba23d9e4"
server_key_iv: "affc21d662c2c384e365a45e0631c35f"
server_key_version: "encryptionkey@0"
I20260619 12:05:01.942265 6985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:05:01.943298 7000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:01.943584 6985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:01.943681 6985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "0dfb4466a7ac441baa3bfd343cbf8704"
format_stamp: "Formatted at 2026-06-19 12:05:01 on dist-test-slave-44vj"
server_key: "af80f2bac5e4d68fd5db0d0dba23d9e4"
server_key_iv: "affc21d662c2c384e365a45e0631c35f"
server_key_version: "encryptionkey@0"
I20260619 12:05:01.943749 6985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:01.959314 6985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:01.962354 6985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:01.962541 6985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:01.962880 6985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:01.963260 6985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:01.963297 6985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:01.963331 6985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:01.963348 6985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:01.970990 6985 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:42535
I20260619 12:05:01.971052 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:42535 every 8 connection(s)
I20260619 12:05:01.971524 6985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:05:01.977768 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 6985
I20260619 12:05:01.977886 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:05:01.978107 814 external_mini_cluster.cc:1428] Setting key 85aad890efcefca5fff127279009f3ce
Jun 19 12:05:01 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:01.984392 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:01.972535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:44451 (local address 127.0.203.190:35585)
0619 12:05:01.972622 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:01.972626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:01.972935 (+ 309us) server_negotiation.cc:407] Connection header received
0619 12:05:01.973482 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:01.973484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:01.973497 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:01.973593 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:01.974766 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:01.975425 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:01.977633 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:01.977940 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:01.978080 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:01.981139 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:01.981142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:01.981142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:01.981154 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:01.982663 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:01.983229 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:01.983231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:01.983231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:01.983294 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:01.983599 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:01.983599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:01.983600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:01.983745 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:01.983809 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:01.984156 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 12:05:01.984266 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:05:01.984649 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:01.972475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:44451)
0619 12:05:01.972705 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:01.972765 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:01.973342 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:01.973669 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:01.973672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:01.974376 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:01.974617 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:01.974623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:01.975622 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:01.975623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:01.977382 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:01.977389 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:01.978096 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:01.978097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:01.978314 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:01.978963 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:01.978977 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:01.981007 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:01.982778 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:01.982794 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:01.982799 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:01.983150 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:01.983389 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:01.983390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:01.983390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:01.983538 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:01.983806 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:01.983815 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:01.984013 (+ 198us) client_negotiation.cc:769] Sending connection context
0619 12:05:01.984128 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 12:05:01.984364 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260619 12:05:01.985316 7114 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35585
I20260619 12:05:01.985468 7114 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:01.985736 7114 heartbeater.cc:511] Master 127.0.203.190:35585 requested a full tablet report, sending...
I20260619 12:05:01.986611 6923 ts_manager.cc:194] Registered new tserver with Master: 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129:42535)
WARNING: no policy specified for oryx/127.0.203.130@KRBTEST.COM; defaulting to no policy
I20260619 12:05:01.987752 6923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:44451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab.
Entry for principal oryx/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab.
I20260619 12:05:02.007292 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35585
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab
--principal=oryx/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:02.107100 7119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:02.107350 7119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:02.107414 7119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:02.108759 7119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:02.108825 7119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:02.108856 7119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:02.111508 7119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab
--principal=oryx/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:02.112104 7119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:02.112531 7119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:02.119220 7125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:02.119468 7119 server_base.cc:1034] running on GCE node
W20260619 12:05:02.119225 7124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.119536 7127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:02.119822 7119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:02.120126 7119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:02.121325 7119 hybrid_clock.cc:648] HybridClock initialized: now 1781870702121314 us; error 34 us; skew 500 ppm
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:02.124439 7119 init.cc:377] Logged in from keytab as oryx/127.0.203.130@KRBTEST.COM (short username oryx)
I20260619 12:05:02.125036 7119 webserver.cc:460] Webserver started at http://127.0.203.130:41345/ using document root <none> and password file <none>
I20260619 12:05:02.125284 7119 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:02.125337 7119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:02.125466 7119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:02.126911 7119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "efbd66f920e54163be6877706e7260a0"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "71bd8e8187dac81e2fb23e1151f113b3"
server_key_iv: "ee6873f0835cd30bbae4ba99c26e8c4e"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.127439 7119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "efbd66f920e54163be6877706e7260a0"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "71bd8e8187dac81e2fb23e1151f113b3"
server_key_iv: "ee6873f0835cd30bbae4ba99c26e8c4e"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.129451 7119 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 12:05:02.130429 7134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.130735 7119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:02.130854 7119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "efbd66f920e54163be6877706e7260a0"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "71bd8e8187dac81e2fb23e1151f113b3"
server_key_iv: "ee6873f0835cd30bbae4ba99c26e8c4e"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.130937 7119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:02.149442 7119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:02.152467 7119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:02.152653 7119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:02.153081 7119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:02.153560 7119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:02.153620 7119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.153656 7119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:02.153671 7119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.161341 7119 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:34055
I20260619 12:05:02.161501 7247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:34055 every 8 connection(s)
I20260619 12:05:02.161826 7119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:02.164012 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 7119
I20260619 12:05:02.164111 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:05:02.164278 814 external_mini_cluster.cc:1428] Setting key 5b97a4abadf0e2340598143b7bdb3999
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.0.203.131@KRBTEST.COM; defaulting to no policy
I20260619 12:05:02.174948 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.162783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:59377 (local address 127.0.203.190:35585)
0619 12:05:02.162910 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.162913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.163150 (+ 237us) server_negotiation.cc:407] Connection header received
0619 12:05:02.163858 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.163860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.163874 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.163938 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.165262 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.166025 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.167796 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.168032 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.168133 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.171411 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.171414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.171414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.171427 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.173237 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.173898 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.173899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.173899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.173967 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.174204 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.174205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.174205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.174345 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.174405 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.174724 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.174840 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 12:05:02.175151 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.162687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:59377)
0619 12:05:02.162927 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.163028 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.163726 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.164041 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.164046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.164878 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.165147 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.165153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.166306 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.166307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.167624 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.167629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.168167 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.168168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.168358 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.169035 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.169049 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.171222 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.173443 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.173456 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.173459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.173801 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.174029 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.174030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.174031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.174147 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.174405 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.174412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.174582 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.174695 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.174907 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260619 12:05:02.176090 7248 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35585
I20260619 12:05:02.176236 7248 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:02.176455 7248 heartbeater.cc:511] Master 127.0.203.190:35585 requested a full tablet report, sending...
I20260619 12:05:02.177204 6923 ts_manager.cc:194] Registered new tserver with Master: efbd66f920e54163be6877706e7260a0 (127.0.203.130:34055)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.131@KRBTEST.COM" created.
I20260619 12:05:02.177886 6923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:59377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab.
Entry for principal oryx/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab.
I20260619 12:05:02.192481 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35585
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab
--principal=oryx/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:02.284291 7253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:02.284593 7253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:02.284624 7253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:02.286144 7253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:02.286219 7253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:02.286254 7253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:02.288235 7253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab
--principal=oryx/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:02.288537 7253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:02.288787 7253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:02.294658 7259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.294713 7258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.294656 7261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:02.295012 7253 server_base.cc:1034] running on GCE node
I20260619 12:05:02.295284 7253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:02.295600 7253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:02.296798 7253 hybrid_clock.cc:648] HybridClock initialized: now 1781870702296773 us; error 46 us; skew 500 ppm
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:02.300046 7253 init.cc:377] Logged in from keytab as oryx/127.0.203.131@KRBTEST.COM (short username oryx)
I20260619 12:05:02.300637 7253 webserver.cc:460] Webserver started at http://127.0.203.131:46191/ using document root <none> and password file <none>
I20260619 12:05:02.300886 7253 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:02.300940 7253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:02.301083 7253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:02.302349 7253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "23961998e5c7406f97ea2e6453f501f3"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "79be76cfa2a6c2454b2de3dba111044f"
server_key_iv: "a28bb682c8502857eba1ff625d3b63f7"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.302738 7253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "23961998e5c7406f97ea2e6453f501f3"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "79be76cfa2a6c2454b2de3dba111044f"
server_key_iv: "a28bb682c8502857eba1ff625d3b63f7"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.304500 7253 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 12:05:02.305619 7268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.305951 7253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:02.306075 7253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "23961998e5c7406f97ea2e6453f501f3"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "79be76cfa2a6c2454b2de3dba111044f"
server_key_iv: "a28bb682c8502857eba1ff625d3b63f7"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.306170 7253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:02.329196 7253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:02.332067 7253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:02.332230 7253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:02.332505 7253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:02.332839 7253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:02.332873 7253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.332907 7253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:02.332931 7253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.340602 7253 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:37085
I20260619 12:05:02.340739 7381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:37085 every 8 connection(s)
I20260619 12:05:02.341086 7253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:02.348945 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 7253
I20260619 12:05:02.349045 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:05:02.349221 814 external_mini_cluster.cc:1428] Setting key 53945ce5888ce86f6107c9f18b3b2e65
I20260619 12:05:02.352602 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.342017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:49793 (local address 127.0.203.190:35585)
0619 12:05:02.342088 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.342091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.342371 (+ 280us) server_negotiation.cc:407] Connection header received
0619 12:05:02.343498 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.343501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.343515 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.343565 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.344303 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.344856 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.345941 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.346136 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.346192 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.349035 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.349037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.349037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.349045 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.350902 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.351471 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.351472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.351472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.351565 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.351844 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.351845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.351845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.351980 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.352040 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.352419 (+ 379us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.352518 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:05:02.352792 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.341927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:49793)
0619 12:05:02.342173 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.342246 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.343326 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.343651 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.343655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.343969 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.344218 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.344223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.344944 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.344945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.345867 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.345870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.346209 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.346210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.346335 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.347015 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.347027 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.348926 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.351002 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.351010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.351012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.351311 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.351647 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.351648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.351648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.351777 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.352088 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.352094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.352272 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.352379 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.352562 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":101,"threads_started":1}
I20260619 12:05:02.353473 7382 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35585
I20260619 12:05:02.353600 7382 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:02.353821 7382 heartbeater.cc:511] Master 127.0.203.190:35585 requested a full tablet report, sending...
I20260619 12:05:02.354259 6923 ts_manager.cc:194] Registered new tserver with Master: 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:02.354662 6923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:49793
I20260619 12:05:02.361856 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.203.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.189.keytab.
Entry for principal oryx/127.0.203.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.189.keytab.
I20260619 12:05:02.392964 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:34321
--webserver_interface=127.0.203.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.189.keytab
--principal=oryx/127.0.203.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:35585,127.0.203.189:34321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:02.486104 7387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:02.486377 7387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:02.486400 7387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:02.487669 7387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:02.487717 7387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:02.487730 7387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:02.487741 7387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:02.487752 7387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:02.489434 7387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:35585,127.0.203.189:34321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.189.keytab
--principal=oryx/127.0.203.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:34321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.203.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
W20260619 12:05:02.489560 7387 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.203.190:35585,127.0.203.189:34321'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 12:05:02.489749 7387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:02.490024 7387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:02.495460 7392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.495501 7393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.495514 7395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:02.495615 7387 server_base.cc:1034] running on GCE node
I20260619 12:05:02.495981 7387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:02.496284 7387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:02.497464 7387 hybrid_clock.cc:648] HybridClock initialized: now 1781870702497446 us; error 47 us; skew 500 ppm
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:02.500957 7387 init.cc:377] Logged in from keytab as oryx/127.0.203.189@KRBTEST.COM (short username oryx)
I20260619 12:05:02.501503 7387 webserver.cc:460] Webserver started at http://127.0.203.189:40009/ using document root <none> and password file <none>
I20260619 12:05:02.501753 7387 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:02.501802 7387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:02.501910 7387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:02.503041 7387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/instance:
uuid: "a5688560fd604d358b3f13b9cc8e3724"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "0657221321cb437df810b8e0552acee9"
server_key_iv: "18ebf60de0d593e89033607ee8bf4f83"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.503417 7387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal/instance:
uuid: "a5688560fd604d358b3f13b9cc8e3724"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "0657221321cb437df810b8e0552acee9"
server_key_iv: "18ebf60de0d593e89033607ee8bf4f83"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.504922 7387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:02.505903 7402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.506189 7387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:02.506301 7387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
uuid: "a5688560fd604d358b3f13b9cc8e3724"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "0657221321cb437df810b8e0552acee9"
server_key_iv: "18ebf60de0d593e89033607ee8bf4f83"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.506387 7387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:02.528460 7387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:02.531229 7387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:02.531443 7387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:02.536758 7387 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.189:34321
I20260619 12:05:02.536834 7454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.189:34321 every 8 connection(s)
I20260619 12:05:02.537222 7387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
I20260619 12:05:02.538499 7455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:02.539489 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 7387
I20260619 12:05:02.541024 7455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.189@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.203.189@KRBTEST.COM
I20260619 12:05:02.552289 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.542467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.0.203.190:35585)
0619 12:05:02.542536 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.542539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.542755 (+ 216us) server_negotiation.cc:407] Connection header received
0619 12:05:02.542841 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.542842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.542854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.542954 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.543619 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.544166 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.545168 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.545375 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.545440 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.548226 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.548228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.548228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.548239 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.549998 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.550626 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.550628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.550628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.550709 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.551252 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.551253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.551253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.551553 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.551632 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.552062 (+ 430us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.552170 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:05:02.552508 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.542363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:58788)
0619 12:05:02.542616 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.542658 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.542769 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.543001 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.543005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.543337 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.543521 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.543528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.544290 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.544292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.545051 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.545054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.545457 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.545458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.545538 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.546051 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.546063 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.548070 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.550140 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.550156 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.550161 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.550544 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.550846 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.550848 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.550848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.551112 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.551666 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.551675 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.551895 (+ 220us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.552002 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.552223 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":131,"threads_started":1}
I20260619 12:05:02.552608 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.540512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.0.203.189:34321)
0619 12:05:02.540790 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.540801 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.540837 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:05:02.541465 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.541475 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.541543 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.541695 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.542488 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.543535 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.544409 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.544954 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.545088 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.548698 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.548716 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.548717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.548727 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.550912 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.551457 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.551461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.551461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.551539 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.551859 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.551860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.551860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.552059 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.552265 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.552363 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.552500 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":124,"threads_started":1}
I20260619 12:05:02.553434 7455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.189@KRBTEST.COM for oryx/127.0.203.189@KRBTEST.COM
I20260619 12:05:02.560989 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.553780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.0.1:59052)
0619 12:05:02.553837 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.553850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.553905 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.554016 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.554016 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.554174 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.554239 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.554241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.554901 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.554902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.555420 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.555421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.555764 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.555765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.555834 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.556278 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.556283 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.557798 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.559765 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.559767 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.559768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.560026 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.560326 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.560328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.560328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.560407 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.560677 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.560678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.560732 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.560805 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.560908 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:05:02.561100 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.553832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.0.203.189:34321)
0619 12:05:02.553889 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.553891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.553899 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:05:02.553958 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.553958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.553967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.554015 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.554305 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.554822 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.555490 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.555704 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.555761 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.558045 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.558047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.558048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.558056 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.559634 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.560143 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.560145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.560145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.560220 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.560481 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.560481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.560482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.560620 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.560677 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.560927 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.561017 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:05:02.562747 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Bootstrap starting.
I20260619 12:05:02.563727 7455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:02.563985 7455 log.cc:824] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:02.565109 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: No bootstrap required, opened a new log
I20260619 12:05:02.567301 7455 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } }
I20260619 12:05:02.567584 7455 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:02.567625 7455 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5688560fd604d358b3f13b9cc8e3724, State: Initialized, Role: FOLLOWER
I20260619 12:05:02.567838 7455 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } }
I20260619 12:05:02.568261 7461 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } } }
I20260619 12:05:02.568372 7455 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:02.568392 7461 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:05:02.570592 7472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:05:02.570658 7472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:05:02.573173 7387 master_runner.cc:428] Detected that this master a5688560fd604d358b3f13b9cc8e3724 is joining an existing cluster
I20260619 12:05:02.573273 7387 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.203.189:34321
I20260619 12:05:02.583704 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.575038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58798 (local address 127.0.203.190:35585)
0619 12:05:02.575222 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.575224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.575305 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:05:02.575401 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.575402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.575415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.575468 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.576151 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.577033 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.578275 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.578520 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.578583 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.580176 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.580177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.580178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.580188 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.581811 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.582574 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.582575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.582576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.582665 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.582949 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.582950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.582950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.583118 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.583191 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.583457 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.583589 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260619 12:05:02.583729 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.574974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.0.203.189:34321)
0619 12:05:02.575175 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.575177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.575384 (+ 207us) server_negotiation.cc:407] Connection header received
0619 12:05:02.575477 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.575478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.575490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.575558 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.577153 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.577932 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.579181 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.579410 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.579475 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.580696 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.580697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.580697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.580705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.582501 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.582856 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.582856 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.582857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.582910 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.583102 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.583102 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.583103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.583228 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.583284 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.583480 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.583579 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260619 12:05:02.583998 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.575006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.0.1:59066)
0619 12:05:02.575294 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.575304 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.575410 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.575568 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.575569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.575873 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.576033 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.576036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.578074 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.578075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.579026 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.579029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.579518 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.579518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.579598 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.580112 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.580117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.580599 (+ 482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.582585 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.582585 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.582586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.582798 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.582969 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.582969 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.582970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.583034 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.583279 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.583280 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.583344 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.583460 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.583641 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":73,"thread_start_us":86,"threads_started":1}
I20260619 12:05:02.584141 7479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.574944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:58798)
0619 12:05:02.575170 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.575184 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.575304 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.575500 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.575501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.575875 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.576033 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.576036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.577164 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.577165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.578189 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.578191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.578598 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.578599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.578661 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.579564 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.579568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.580042 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.581919 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.581920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.581921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.582478 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.582739 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.582739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.582740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.582816 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.583182 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.583183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.583251 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.583864 (+ 613us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.584011 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":15,"thread_start_us":140,"threads_started":1}
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.190@KRBTEST.COM for oryx/127.0.203.189@KRBTEST.COM
I20260619 12:05:02.596201 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.586936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.0.203.189:34321)
0619 12:05:02.586994 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.586996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.587203 (+ 207us) server_negotiation.cc:407] Connection header received
0619 12:05:02.587304 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.587305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.587318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.587407 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.587980 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.588803 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.589809 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.590169 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.590301 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.593096 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.593099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.593099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.593109 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.594612 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.595247 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.595248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.595248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.595309 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.595560 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.595560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.595561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.595694 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.595751 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.596033 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.596116 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:05:02.596287 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.586858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.0.1:59070)
0619 12:05:02.587093 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.587129 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.587202 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.587430 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.587434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.587744 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.587817 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.587819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.588953 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.588954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.589665 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.589667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.590299 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.590300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.590381 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.590974 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.590979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.592911 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.594730 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.594744 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.594748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.595130 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.595416 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.595417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.595417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.595494 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.595762 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.595771 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.595861 (+ 90us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.595969 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.596140 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":99,"threads_started":1}
I20260619 12:05:02.597205 6923 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.203.189:34321: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724"
member_type: NON_VOTER
last_known_addr {
host: "127.0.203.189"
port: 34321
}
attrs {
promote: true
}
}
dest_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa"
cas_config_opid_index: -1
I20260619 12:05:02.597594 6923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } }
I20260619 12:05:02.598146 7482 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } }
I20260619 12:05:02.598568 7429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:02.599525 7482 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.203.189:34321
I20260619 12:05:02.599634 7482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } }
I20260619 12:05:02.599704 7482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:02.600201 7429 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Refusing update from remote peer 5843ca67bb6f4ea1ad6ddc7965a21eaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 12:05:02.600541 7387 master.cc:537] Master@127.0.203.189:34321 shutting down...
I20260619 12:05:02.601059 7481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:05:02.602716 6908 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 12:05:02.603330 6908 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa -> Peer a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189:34321): Couldn't send request to peer a5688560fd604d358b3f13b9cc8e3724. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 12:05:02.603776 7387 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:02.603897 7387 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:02.603961 7387 tablet_replica.cc:330] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: stopping tablet replica
I20260619 12:05:02.617205 7387 master.cc:552] Master@127.0.203.189:34321 shutdown complete.
I20260619 12:05:02.617280 7387 master_runner.cc:305] Clearing existing system tablet
I20260619 12:05:02.617416 7387 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:02.617466 7387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:02.618187 7387 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260619 12:05:02.618629 7486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.618830 7387 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:02.618965 7387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:05:02.618999 7387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
uuid: "a5688560fd604d358b3f13b9cc8e3724"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "0657221321cb437df810b8e0552acee9"
server_key_iv: "18ebf60de0d593e89033607ee8bf4f83"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.620087 7387 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:02.622150 7387 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 12:05:02.622262 7387 log.cc:1196] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 12:05:02.622476 7387 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Deleting consensus metadata
I20260619 12:05:02.622874 7387 master_runner.cc:315] Copying system tablet from 127.0.203.190:35585
I20260619 12:05:02.623734 7387 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: tablet copy: Beginning tablet copy session from remote peer at address 127.0.203.190:35585
I20260619 12:05:02.631417 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.624299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.0.203.190:35585)
0619 12:05:02.624367 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.624370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.624511 (+ 141us) server_negotiation.cc:407] Connection header received
0619 12:05:02.624602 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.624603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.624614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.624674 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.625080 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.625627 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.626711 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.626926 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.626993 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.628259 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.628261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.628261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.628269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.629712 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.630409 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.630410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.630411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.630474 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.630742 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.630743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.630744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.630925 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.630991 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.631199 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.631278 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:05:02.631523 7492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.624197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:58806)
0619 12:05:02.624401 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.624416 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.624520 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.624701 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.624702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.624921 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.624998 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.625000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.625714 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.625715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.626597 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.626600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.627012 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.627013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.627090 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.627699 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.627704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.628147 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.629829 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.629830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.629831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.630295 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.630557 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.630557 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.630558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.630652 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.630998 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.630999 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.631059 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.631143 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.631265 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":116,"threads_started":1}
I20260619 12:05:02.632210 6943 tablet_copy_service.cc:140] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a5688560fd604d358b3f13b9cc8e3724 ({username='oryx', principal='oryx/127.0.203.189@KRBTEST.COM'} at 127.0.0.1:58806)
I20260619 12:05:02.632325 6943 tablet_copy_service.cc:161] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a5688560fd604d358b3f13b9cc8e3724 at {username='oryx', principal='oryx/127.0.203.189@KRBTEST.COM'} at 127.0.0.1:58806: session id = a5688560fd604d358b3f13b9cc8e3724-00000000000000000000000000000000
I20260619 12:05:02.633137 6943 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 12:05:02.634064 7387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 12:05:02.635618 7387 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: tablet copy: Starting download of 0 data blocks...
I20260619 12:05:02.635838 7387 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: tablet copy: Starting download of 1 WAL segments...
I20260619 12:05:02.636914 7387 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 12:05:02.638000 6943 tablet_copy_service.cc:342] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Request end of tablet copy session a5688560fd604d358b3f13b9cc8e3724-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.203.189@KRBTEST.COM'} at 127.0.0.1:58806
I20260619 12:05:02.638068 6943 tablet_copy_service.cc:433] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: ending tablet copy session a5688560fd604d358b3f13b9cc8e3724-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a5688560fd604d358b3f13b9cc8e3724
I20260619 12:05:02.641438 7387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:02.642964 7493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.643015 7496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.643146 7494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:02.643537 7387 server_base.cc:1034] running on GCE node
I20260619 12:05:02.643646 7387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:02.643813 7387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:02.644927 7387 hybrid_clock.cc:648] HybridClock initialized: now 1781870702644904 us; error 48 us; skew 500 ppm
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:02.646589 7387 init.cc:377] Logged in from keytab as oryx/127.0.203.189@KRBTEST.COM (short username oryx)
I20260619 12:05:02.646996 7387 webserver.cc:460] Webserver started at http://127.0.203.189:40583/ using document root <none> and password file <none>
I20260619 12:05:02.647105 7387 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:02.647135 7387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:02.647868 7387 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:02.648417 7503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.648528 7387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:05:02.648572 7387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
uuid: "a5688560fd604d358b3f13b9cc8e3724"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "0657221321cb437df810b8e0552acee9"
server_key_iv: "18ebf60de0d593e89033607ee8bf4f83"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.648638 7387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:02.670490 7387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:02.672190 7387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:02.675917 7387 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.189:34321
I20260619 12:05:02.676167 7555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.189:34321 every 8 connection(s)
I20260619 12:05:02.676331 7387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
I20260619 12:05:02.677157 7556 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:05:02.677779 7556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Bootstrap starting.
I20260619 12:05:02.680589 7556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:05:02.680790 7556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: Bootstrap complete.
I20260619 12:05:02.681010 7556 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } }
I20260619 12:05:02.681092 7556 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a5688560fd604d358b3f13b9cc8e3724, State: Initialized, Role: LEARNER
I20260619 12:05:02.681118 7556 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } }
I20260619 12:05:02.681305 7556 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:02.681355 7559 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } }
I20260619 12:05:02.681530 7559 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:05:02.682895 7570 catalog_manager.cc:1259] Loaded cluster ID: d887bc3a8d6d40df98ced7921a80c4f8
I20260619 12:05:02.682946 7570 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: loading cluster ID for follower catalog manager: success
I20260619 12:05:02.684110 7570 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: acquiring CA information for follower catalog manager: success
I20260619 12:05:02.684423 7570 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:05:02.713419 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.706767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.0.203.189:34321)
0619 12:05:02.706989 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.706992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.707000 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:05:02.707027 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.707028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.707040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.707107 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.707791 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.708539 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.709370 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.709591 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.709647 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.710944 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.710945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.710946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.710953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.712348 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.712686 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.712687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.712687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.712741 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.712907 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.712908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.712909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.713026 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.713122 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.713213 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.713302 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for oryx/127.0.203.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.188.keytab.
Entry for principal oryx/127.0.203.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.188.keytab.
I20260619 12:05:02.743675 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:38511
--webserver_interface=127.0.203.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.188.keytab
--principal=oryx/127.0.203.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:02.842425 7574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:02.842674 7574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:02.842698 7574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:02.843998 7574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:02.844065 7574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:02.844079 7574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:02.844091 7574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:02.844102 7574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:02.845918 7574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.188.keytab
--principal=oryx/127.0.203.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:38511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.203.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:02.846176 7574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:02.846402 7574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:02.852187 7580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:02.852140 7582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:02.852250 7574 server_base.cc:1034] running on GCE node
W20260619 12:05:02.852140 7579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:02.852753 7574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:02.853165 7574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:02.854317 7574 hybrid_clock.cc:648] HybridClock initialized: now 1781870702854293 us; error 55 us; skew 500 ppm
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:02.857750 7574 init.cc:377] Logged in from keytab as oryx/127.0.203.188@KRBTEST.COM (short username oryx)
I20260619 12:05:02.858268 7574 webserver.cc:460] Webserver started at http://127.0.203.188:35899/ using document root <none> and password file <none>
I20260619 12:05:02.858530 7574 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:02.858579 7574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:02.858713 7574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:02.859982 7574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/instance:
uuid: "622b770926624591a1afcea86866d2e9"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "97ec0fdac371b9130eb15e7341d9fdf9"
server_key_iv: "6dd3c8c4d9e0d53b9bdc1831abf86975"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.860374 7574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal/instance:
uuid: "622b770926624591a1afcea86866d2e9"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "97ec0fdac371b9130eb15e7341d9fdf9"
server_key_iv: "6dd3c8c4d9e0d53b9bdc1831abf86975"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.861972 7574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:05:02.862983 7589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.863226 7574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:02.863386 7574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
uuid: "622b770926624591a1afcea86866d2e9"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "97ec0fdac371b9130eb15e7341d9fdf9"
server_key_iv: "6dd3c8c4d9e0d53b9bdc1831abf86975"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.863467 7574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:02.883991 7574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:02.886420 7574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:02.886562 7574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:02.891036 7574 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.188:38511
I20260619 12:05:02.891113 7641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.188:38511 every 8 connection(s)
I20260619 12:05:02.891536 7574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
I20260619 12:05:02.892693 7642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:02.894356 7642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.188@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:02.900689 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 7574
I20260619 12:05:02.904142 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.895215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.0.203.190:35585)
0619 12:05:02.895297 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.895298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.895479 (+ 181us) server_negotiation.cc:407] Connection header received
0619 12:05:02.896001 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.896002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.896013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.896089 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.896748 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.897284 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.898406 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.898595 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.898652 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.901110 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.901112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.901112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.901120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.902671 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.903144 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.903145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.903145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.903202 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.903469 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.903470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.903470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.903604 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.903670 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.903960 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.904046 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:05:02.904345 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.895108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:58818)
0619 12:05:02.895305 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.895352 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.895903 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.896167 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.896177 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.896457 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.896664 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.896668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.897396 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.897397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.898292 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.898294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.898679 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.898680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.898790 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.899344 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.899353 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.901012 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.902777 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.902787 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.902790 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.903068 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.903259 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.903260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.903260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.903417 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.903661 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.903667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.903823 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.903911 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.904093 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260619 12:05:02.904960 7642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.203.188@KRBTEST.COM
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.188@KRBTEST.COM for oryx/127.0.203.189@KRBTEST.COM
I20260619 12:05:02.910609 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.901223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38200 (local address 127.0.203.188:38511)
0619 12:05:02.901433 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.901438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.901457 (+ 19us) server_negotiation.cc:407] Connection header received
0619 12:05:02.901477 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.901479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.901493 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.901563 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.902154 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.902885 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.903655 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.904007 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.904067 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.907122 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.907134 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.907134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.907145 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.909005 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.909544 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.909547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.909547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.909622 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.909920 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.909920 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.909921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.910138 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.910318 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.910396 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.910484 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260619 12:05:02.912945 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.905255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.0.1:59098)
0619 12:05:02.905343 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.905354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.905392 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.905570 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.905571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.905723 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.905788 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.905789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.906769 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.906770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.907529 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.907532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.907986 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.907987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.908052 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.908463 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.908467 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.909916 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.911802 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.911803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.911804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.911979 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.912219 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.912220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.912220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.912266 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.912629 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.912630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.912692 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.912771 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.912870 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260619 12:05:02.913125 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.905327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59098 (local address 127.0.203.189:34321)
0619 12:05:02.905383 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.905385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.905396 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:02.905450 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.905450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.905461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.905516 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.905900 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.906656 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.907616 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.907887 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.907980 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.910009 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.910010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.910011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.910020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.911697 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.912075 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.912077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.912077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.912142 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.912349 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.912350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.912351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.912507 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.912584 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.912832 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.912993 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:05:02.913869 7642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870702, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.188@KRBTEST.COM for oryx/127.0.203.188@KRBTEST.COM
I20260619 12:05:02.919979 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.914137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.0.1:38214)
0619 12:05:02.914203 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.914213 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.914253 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.914364 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.914365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.914502 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.914537 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.914538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.915154 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.915155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.915698 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.915700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.916031 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.916032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.916074 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.916480 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.916485 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.917635 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.918962 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.918963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.918963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.919127 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.919298 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.919299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.919299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.919353 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.919631 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.919631 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.919669 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.919733 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.919838 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 12:05:02.919984 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.914218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38214 (local address 127.0.203.188:38511)
0619 12:05:02.914276 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.914278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.914285 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:05:02.914300 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.914301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.914311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.914361 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.914596 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.915083 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.915766 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.915977 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.916032 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.917711 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.917712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.917713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.917719 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.918871 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.919191 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.919192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.919192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.919244 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.919444 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.919444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.919445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.919572 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.919635 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.919769 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.919886 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":41}
I20260619 12:05:02.921788 7642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Bootstrap starting.
I20260619 12:05:02.922837 7642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:02.923164 7642 log.cc:824] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:02.924115 7642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: No bootstrap required, opened a new log
I20260619 12:05:02.926538 7642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } }
I20260619 12:05:02.926698 7642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:02.926721 7642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622b770926624591a1afcea86866d2e9, State: Initialized, Role: FOLLOWER
I20260619 12:05:02.926854 7642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } }
I20260619 12:05:02.927235 7642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:02.927237 7647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } } }
I20260619 12:05:02.927477 7647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:05:02.929133 7658 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:05:02.929208 7658 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:05:02.932487 7574 master_runner.cc:428] Detected that this master 622b770926624591a1afcea86866d2e9 is joining an existing cluster
I20260619 12:05:02.932536 7574 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.203.188:38511
I20260619 12:05:02.960105 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.933646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.0.203.190:35585)
0619 12:05:02.933747 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.933749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.933905 (+ 156us) server_negotiation.cc:407] Connection header received
0619 12:05:02.934006 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.934007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.934020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.934052 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.934660 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.935522 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.936543 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.936844 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.936878 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.956637 (+ 19759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.956641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.956641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.956653 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.958460 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959213 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.959214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.959215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.959279 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959516 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.959517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.959518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.959673 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.959723 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.959939 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.960014 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 12:05:02.960103 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.933964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.0.203.188:38511)
0619 12:05:02.934000 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.934002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.934093 (+ 91us) server_negotiation.cc:407] Connection header received
0619 12:05:02.934162 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.934163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.934171 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.934194 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.934640 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.935442 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.936537 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.936876 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.936962 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.956737 (+ 19775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.956738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.956738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.956744 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.958874 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959248 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.959249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.959249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.959294 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959524 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.959525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.959525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.959673 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.959715 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.959895 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.959983 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":20}
I20260619 12:05:02.960373 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.933828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.0.203.189:34321)
0619 12:05:02.933878 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.933879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.934230 (+ 351us) server_negotiation.cc:407] Connection header received
0619 12:05:02.934334 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.934335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.934346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.934375 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.934942 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.935744 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.937215 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.937833 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.937918 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.956613 (+ 18695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.956616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.956616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.956625 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.958977 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959504 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.959505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.959506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.959579 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959788 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.959789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.959789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.959931 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.959973 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.960175 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.960268 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:05:02.960503 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.933575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:58826)
0619 12:05:02.933832 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.933845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.933933 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.934126 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.934127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.934454 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.934565 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.934566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.935650 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.935651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.936465 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.936466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.936971 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.936971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.937046 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.938032 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.938038 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.956459 (+ 18421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.958581 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.958583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.958583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.959129 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.959387 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.959388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.959445 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.959765 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.959765 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.959812 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.960258 (+ 446us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.960395 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260619 12:05:02.960906 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.933819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.0.1:38226)
0619 12:05:02.934043 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.934050 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.934111 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.934243 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.934244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.934454 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.934562 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.934565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.935523 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.935523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.936448 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.936451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.936977 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.936977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.937046 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.956115 (+ 19069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.956124 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.956640 (+ 516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.958937 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.958937 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.958938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.959182 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.959413 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959413 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.959414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.959459 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.959754 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.959756 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.959797 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.960531 (+ 734us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.960813 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":20,"thread_start_us":68,"threads_started":1}
I20260619 12:05:02.961385 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.933945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.0.1:59108)
0619 12:05:02.934160 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.934172 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.934262 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.934459 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.934460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.934732 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.934854 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.934856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.935862 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.935863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.937100 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.937102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.937935 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.937936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.938020 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.938421 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.938426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.956434 (+ 18008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.959119 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959121 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.959122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.959417 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.959648 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.959649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.959649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.959712 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.960018 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.960019 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.960072 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.960923 (+ 851us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.961258 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":4,"thread_start_us":120,"threads_started":1}
I20260619 12:05:02.962390 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6985
I20260619 12:05:02.969748 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:42535
--local_ip_for_outbound_sockets=127.0.203.129
--tserver_master_addrs=127.0.203.190:35585
--webserver_port=46879
--webserver_interface=127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab
--principal=oryx/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:05:02 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870701, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.190@KRBTEST.COM for oryx/127.0.203.188@KRBTEST.COM
I20260619 12:05:02.976626 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.964281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.0.1:38242)
0619 12:05:02.967603 (+ 3322us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.967623 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.967734 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.968005 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.968007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.968223 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:02.968299 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.968301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.969008 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.969009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.969694 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.969698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.970347 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.970349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.970446 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.971021 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:02.971026 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:02.972830 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:02.974955 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.974957 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.974958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.975255 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.975681 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:02.975682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:02.975682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:02.975766 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:02.976178 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:02.976179 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:02.976252 (+ 73us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.976334 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.976493 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3282}
I20260619 12:05:02.977061 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.964329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.0.203.188:38511)
0619 12:05:02.964383 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.964385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.967742 (+ 3357us) server_negotiation.cc:407] Connection header received
0619 12:05:02.967884 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.967885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.967900 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.967970 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:02.968388 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.968931 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.969824 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.970168 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.970255 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.972969 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:02.972971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:02.972972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:02.972981 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:02.974826 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.975355 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.975387 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.975388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.975467 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:02.975865 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:02.975867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:02.975868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:02.976098 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:02.976188 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.976457 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.976584 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:05:02.978026 6923 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.203.188:38511: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "622b770926624591a1afcea86866d2e9"
member_type: NON_VOTER
last_known_addr {
host: "127.0.203.188"
port: 38511
}
attrs {
promote: true
}
}
dest_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa"
cas_config_opid_index: 5
I20260619 12:05:02.978452 6923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } }
I20260619 12:05:02.978972 7482 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 622b770926624591a1afcea86866d2e9 (127.0.203.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.980211 7482 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.203.188:38511
I20260619 12:05:02.980309 7482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.980378 7482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:02.980722 7574 master.cc:537] Master@127.0.203.188:38511 shutting down...
W20260619 12:05:02.981793 6909 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa -> Peer 622b770926624591a1afcea86866d2e9 (127.0.203.188:38511): Couldn't send request to peer 622b770926624591a1afcea86866d2e9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 12:05:02.985038 7574 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:02.985172 7574 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:02.985226 7574 tablet_replica.cc:330] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: stopping tablet replica
I20260619 12:05:02.986984 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.979100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.0.1:59122)
0619 12:05:02.979288 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.979302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:02.979346 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:02.982952 (+ 3606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:02.982954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:02.982957 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:02.983023 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.983025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.984044 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.984045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.985737 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:02.985751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.986647 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.986648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:02.986729 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.986761 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 12:05:02.986822 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:05:02.986883 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260619 12:05:02.987638 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.979241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59122 (local address 127.0.203.189:34321)
0619 12:05:02.982387 (+ 3146us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:02.982389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:02.982403 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:05:02.982433 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:02.982434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:02.982462 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:02.982557 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:02.983152 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.983944 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.985853 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:02.986450 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:02.986507 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:02.986582 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:02.987266 (+ 684us) server_negotiation.cc:299] Negotiation successful
0619 12:05:02.987315 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3115}
I20260619 12:05:02.988160 7530 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 LEARNER]: Refusing update from remote peer 5843ca67bb6f4ea1ad6ddc7965a21eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 12:05:02.988377 7481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 12:05:02.989020 7530 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 622b770926624591a1afcea86866d2e9 (127.0.203.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.990216 7559 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5843ca67bb6f4ea1ad6ddc7965a21eaa. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.990295 7559 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:05:02.990326 7559 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: NON_VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: true } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.990360 7559 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:05:02.991142 7481 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: attempting to promote NON_VOTER a5688560fd604d358b3f13b9cc8e3724 to VOTER
I20260619 12:05:02.991410 7481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } }
I20260619 12:05:02.992087 7530 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 LEARNER]: Refusing update from remote peer 5843ca67bb6f4ea1ad6ddc7965a21eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 12:05:02.992133 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.991955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.0.1:38258)
0619 12:05:02.992025 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.992081 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:38511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 12:05:02.992350 7482 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 12:05:02.992444 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:02.992369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.0.1:38274)
0619 12:05:02.992401 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:02.992421 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:38511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 12:05:02.992496 6909 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa -> Peer 622b770926624591a1afcea86866d2e9 (127.0.203.188:38511): Couldn't send request to peer 622b770926624591a1afcea86866d2e9. Status: Network error: Client connection negotiation failed: client connection to 127.0.203.188:38511: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 12:05:02.992854 7574 master.cc:552] Master@127.0.203.188:38511 shutdown complete.
I20260619 12:05:02.992906 7574 master_runner.cc:305] Clearing existing system tablet
I20260619 12:05:02.993050 7574 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:02.993103 7574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:02.993201 7482 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.994176 7482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.994246 7482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:02.994298 7482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.994338 7482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:02.995038 7530 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.995509 7574 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:05:02.996443 7669 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:02.996553 7669 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:05:02.996656 7672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:02.996774 7574 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:02.996827 7574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:05:02.996857 7574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
uuid: "622b770926624591a1afcea86866d2e9"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "97ec0fdac371b9130eb15e7341d9fdf9"
server_key_iv: "6dd3c8c4d9e0d53b9bdc1831abf86975"
server_key_version: "encryptionkey@0"
I20260619 12:05:02.997727 7574 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:02.999192 7574 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 12:05:02.999269 7574 log.cc:1196] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 12:05:02.999500 7574 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Deleting consensus metadata
I20260619 12:05:02.999702 7574 master_runner.cc:315] Copying system tablet from 127.0.203.190:35585
I20260619 12:05:03.000476 7574 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: tablet copy: Beginning tablet copy session from remote peer at address 127.0.203.190:35585
I20260619 12:05:03.009228 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.000830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:58830)
0619 12:05:03.001023 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.001040 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.001129 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.001322 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.001324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.001533 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.001613 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.001615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.002629 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.002631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.003478 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.003481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.004025 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.004026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.004095 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.004779 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.004785 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.005212 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.007838 (+ 2626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.007839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.007840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.008130 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.008405 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.008405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.008406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.008491 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.008815 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.008816 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.008888 (+ 72us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.008967 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.009107 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260619 12:05:03.009660 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.001053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.0.203.190:35585)
0619 12:05:03.001104 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.001106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.001120 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:05:03.001226 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.001227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.001240 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.001301 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.001704 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.002532 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.003614 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.003932 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.004016 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.005314 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.005316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.005316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.005325 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.007700 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.008241 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.008242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.008243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.008324 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.008568 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.008569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.008569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.008730 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.008804 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.009021 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.009123 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:05:03.010188 6943 tablet_copy_service.cc:140] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 622b770926624591a1afcea86866d2e9 ({username='oryx', principal='oryx/127.0.203.188@KRBTEST.COM'} at 127.0.0.1:58830)
I20260619 12:05:03.010258 6943 tablet_copy_service.cc:161] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 622b770926624591a1afcea86866d2e9 at {username='oryx', principal='oryx/127.0.203.188@KRBTEST.COM'} at 127.0.0.1:58830: session id = 622b770926624591a1afcea86866d2e9-00000000000000000000000000000000
I20260619 12:05:03.011013 6943 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 12:05:03.011664 7574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 12:05:03.013191 7574 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: tablet copy: Starting download of 0 data blocks...
I20260619 12:05:03.013387 7574 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: tablet copy: Starting download of 1 WAL segments...
I20260619 12:05:03.014520 7574 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 12:05:03.015703 6943 tablet_copy_service.cc:342] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Request end of tablet copy session 622b770926624591a1afcea86866d2e9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.203.188@KRBTEST.COM'} at 127.0.0.1:58830
I20260619 12:05:03.015782 6943 tablet_copy_service.cc:433] P 5843ca67bb6f4ea1ad6ddc7965a21eaa: ending tablet copy session 622b770926624591a1afcea86866d2e9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 622b770926624591a1afcea86866d2e9
I20260619 12:05:03.019668 7574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:03.021175 7679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:03.024462 7680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:03.024814 7574 server_base.cc:1034] running on GCE node
W20260619 12:05:03.027511 7682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:03.027819 7574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:03.028074 7574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:03.029206 7574 hybrid_clock.cc:648] HybridClock initialized: now 1781870703029178 us; error 56 us; skew 500 ppm
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:03.030948 7574 init.cc:377] Logged in from keytab as oryx/127.0.203.188@KRBTEST.COM (short username oryx)
I20260619 12:05:03.031302 7574 webserver.cc:460] Webserver started at http://127.0.203.188:39475/ using document root <none> and password file <none>
I20260619 12:05:03.031509 7574 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:03.031548 7574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:03.032508 7574 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:03.033223 7689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.033511 7574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:03.033567 7574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
uuid: "622b770926624591a1afcea86866d2e9"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "97ec0fdac371b9130eb15e7341d9fdf9"
server_key_iv: "6dd3c8c4d9e0d53b9bdc1831abf86975"
server_key_version: "encryptionkey@0"
I20260619 12:05:03.033631 7574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:03.067306 7574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:03.069566 7574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:03.073733 7574 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.188:38511
I20260619 12:05:03.073899 7741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.188:38511 every 8 connection(s)
I20260619 12:05:03.074146 7574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
I20260619 12:05:03.075114 7742 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:05:03.075690 7742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Bootstrap starting.
I20260619 12:05:03.079200 7742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:05:03.079476 7742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Bootstrap complete.
I20260619 12:05:03.079701 7742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } }
I20260619 12:05:03.079800 7742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 622b770926624591a1afcea86866d2e9, State: Initialized, Role: LEARNER
I20260619 12:05:03.079833 7742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } }
I20260619 12:05:03.080018 7742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:03.080202 7745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:03.080448 7745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:05:03.083072 7756 catalog_manager.cc:1259] Loaded cluster ID: d887bc3a8d6d40df98ced7921a80c4f8
I20260619 12:05:03.083196 7756 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: loading cluster ID for follower catalog manager: success
I20260619 12:05:03.084470 7756 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: acquiring CA information for follower catalog manager: success
I20260619 12:05:03.084932 7756 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 12:05:03.090584 7667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:03.090857 7667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:03.090900 7667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:03.092598 7667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:03.092682 7667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:03.092738 7667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:03.094662 7667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab
--principal=oryx/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:42535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=46879
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:03.094985 7667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:03.095240 7667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:03.100971 7764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:03.101058 7761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:03.100971 7762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:03.101392 7667 server_base.cc:1034] running on GCE node
I20260619 12:05:03.101720 7667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:03.101984 7667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:03.103188 7667 hybrid_clock.cc:648] HybridClock initialized: now 1781870703103123 us; error 63 us; skew 500 ppm
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:03.106169 7667 init.cc:377] Logged in from keytab as oryx/127.0.203.129@KRBTEST.COM (short username oryx)
I20260619 12:05:03.106724 7667 webserver.cc:460] Webserver started at http://127.0.203.129:46879/ using document root <none> and password file <none>
I20260619 12:05:03.106972 7667 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:03.107043 7667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:03.108675 7667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:03.109407 7771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.109638 7667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:03.109838 7667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "0dfb4466a7ac441baa3bfd343cbf8704"
format_stamp: "Formatted at 2026-06-19 12:05:01 on dist-test-slave-44vj"
server_key: "af80f2bac5e4d68fd5db0d0dba23d9e4"
server_key_iv: "affc21d662c2c384e365a45e0631c35f"
server_key_version: "encryptionkey@0"
I20260619 12:05:03.110371 7667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:03.136225 7667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:03.138244 7667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:03.138389 7667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:03.138612 7667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:03.138942 7667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:03.138984 7667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.139014 7667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:03.139031 7667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.146379 7667 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:42535
I20260619 12:05:03.146479 7884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:42535 every 8 connection(s)
I20260619 12:05:03.146857 7667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:05:03.151340 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 7667
I20260619 12:05:03.151577 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 7119
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.188@KRBTEST.COM
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.189@KRBTEST.COM
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:03.160873 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.147891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:60285 (local address 127.0.203.190:35585)
0619 12:05:03.147966 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.147969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.148357 (+ 388us) server_negotiation.cc:407] Connection header received
0619 12:05:03.149201 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.149203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.149215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.149267 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.150346 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.151158 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.152958 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.153245 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.153284 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.157622 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.157624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.157625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.157633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.159026 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.159649 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.159650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.159650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.159724 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.160031 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.160032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.160032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.160195 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.160252 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.160667 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.160764 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:05:03.160873 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.147816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:46209 (local address 127.0.203.189:34321)
0619 12:05:03.147879 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.147882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.148216 (+ 334us) server_negotiation.cc:407] Connection header received
0619 12:05:03.149124 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.149126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.149140 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.149201 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.150312 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.151156 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.152812 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.153110 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.153150 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.157160 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.157162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.157163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.157173 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.158939 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.159649 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.159650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.159650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.159724 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.160031 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.160032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.160032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.160204 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.160252 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.160667 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.160764 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:05:03.160863 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:34055
--local_ip_for_outbound_sockets=127.0.203.130
--tserver_master_addrs=127.0.203.190:35585
--webserver_port=41345
--webserver_interface=127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab
--principal=oryx/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:05:03.161533 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.147825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:60285)
0619 12:05:03.148258 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.148272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.148951 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.149729 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.149731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.149951 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.150194 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.150201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.151272 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.151274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.152867 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.152869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.153358 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.153359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.153420 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.154178 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.154191 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.157539 (+ 3348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.159123 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.159123 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.159124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.159557 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.159806 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.159806 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.159806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.159953 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.160282 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.160289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.160481 (+ 192us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.160995 (+ 514us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.161246 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":13,"thread_start_us":114,"threads_started":1}
I20260619 12:05:03.161535 7889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.148095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.203.129:38149)
0619 12:05:03.148247 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.148255 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.148912 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.149391 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.149392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.149931 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.150194 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.150201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.151218 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.151219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.152686 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.152690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.153184 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.153185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.153353 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.154231 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.154237 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.158052 (+ 3815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.160289 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.160291 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.160292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.160524 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.160749 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.160750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.160750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.160818 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.161121 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.161122 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.161171 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.161214 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.161356 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":48,"thread_start_us":74,"threads_started":1}
I20260619 12:05:03.162061 7890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.147773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.203.129:46209)
0619 12:05:03.148046 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.148102 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.148898 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.149375 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.149380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.149900 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.150191 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.150199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.151272 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.151274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.152686 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.152691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.153241 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.153242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.153359 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.154178 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.154191 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.156994 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.159051 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.159064 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.159067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.159557 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.159787 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.159787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.159788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.159953 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.160293 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.160294 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.160481 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.161581 (+ 1100us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.161988 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":11,"thread_start_us":103,"threads_started":1}
I20260619 12:05:03.162123 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.148177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:38149 (local address 127.0.203.188:38511)
0619 12:05:03.148543 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.148545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.148554 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:05:03.149127 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.149128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.149146 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.149247 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.150339 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.151044 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.152799 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.153096 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.153667 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.158533 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.158535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.158536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.158545 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.160199 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.160619 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.160620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.160620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.160686 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.160888 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.160889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.160889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.161043 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.161875 (+ 832us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.161919 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.162023 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":77,"threads_started":1}
I20260619 12:05:03.162706 7885 heartbeater.cc:347] Connected to a master server at 127.0.203.188:38511
I20260619 12:05:03.162827 7885 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.163003 7886 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35585
I20260619 12:05:03.163022 7885 heartbeater.cc:511] Master 127.0.203.188:38511 requested a full tablet report, sending...
I20260619 12:05:03.163038 7886 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.163089 7886 heartbeater.cc:511] Master 127.0.203.190:35585 requested a full tablet report, sending...
I20260619 12:05:03.163252 7887 heartbeater.cc:347] Connected to a master server at 127.0.203.189:34321
I20260619 12:05:03.163295 7887 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.163417 7887 heartbeater.cc:511] Master 127.0.203.189:34321 requested a full tablet report, sending...
I20260619 12:05:03.163584 6923 ts_manager.cc:194] Re-registered known tserver with Master: 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129:42535)
I20260619 12:05:03.163739 7706 ts_manager.cc:194] Registered new tserver with Master: 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129:42535)
I20260619 12:05:03.164114 6923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:60285
I20260619 12:05:03.164168 7520 ts_manager.cc:194] Registered new tserver with Master: 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129:42535)
W20260619 12:05:03.251587 7893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:03.251850 7893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:03.251880 7893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:03.253175 7893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:03.253233 7893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:03.253263 7893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:03.254851 7893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab
--principal=oryx/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:34055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=41345
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:03.255128 7893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:03.255421 7893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:03.260892 7899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:03.260892 7901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:03.260917 7898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:03.261088 7893 server_base.cc:1034] running on GCE node
I20260619 12:05:03.261382 7893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:03.261648 7893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:03.262800 7893 hybrid_clock.cc:648] HybridClock initialized: now 1781870703262792 us; error 31 us; skew 500 ppm
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:03.265661 7893 init.cc:377] Logged in from keytab as oryx/127.0.203.130@KRBTEST.COM (short username oryx)
I20260619 12:05:03.266178 7893 webserver.cc:460] Webserver started at http://127.0.203.130:41345/ using document root <none> and password file <none>
I20260619 12:05:03.266428 7893 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:03.266494 7893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:03.268111 7893 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:05:03.268939 7908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.269192 7893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:03.269292 7893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "efbd66f920e54163be6877706e7260a0"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "71bd8e8187dac81e2fb23e1151f113b3"
server_key_iv: "ee6873f0835cd30bbae4ba99c26e8c4e"
server_key_version: "encryptionkey@0"
I20260619 12:05:03.269714 7893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:03.291115 7893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:03.293536 7893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:03.293687 7893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:03.293962 7893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:03.294293 7893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:03.294327 7893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.294358 7893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:03.294374 7893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.301508 7893 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:34055
I20260619 12:05:03.301643 8021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:34055 every 8 connection(s)
I20260619 12:05:03.301985 7893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:03.308003 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 7893
I20260619 12:05:03.308185 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 7253
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.189@KRBTEST.COM
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.188@KRBTEST.COM
I20260619 12:05:03.314728 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:37085
--local_ip_for_outbound_sockets=127.0.203.131
--tserver_master_addrs=127.0.203.190:35585
--webserver_port=46191
--webserver_interface=127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab
--principal=oryx/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:05:03.316226 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.302863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:47163 (local address 127.0.203.189:34321)
0619 12:05:03.302928 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.302931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.303272 (+ 341us) server_negotiation.cc:407] Connection header received
0619 12:05:03.304539 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.304542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.304557 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.304635 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.306565 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.307080 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308305 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.308581 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308621 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.312608 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.312610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.312611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.312622 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.314217 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.314748 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.314749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.314750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.314805 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.315324 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.315325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.315326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.315515 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.315571 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.316012 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.316109 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:05:03.317092 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.302876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.203.130:47163)
0619 12:05:03.303111 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.303153 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.304389 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.304706 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.304711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.305191 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.305495 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.305502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.307191 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.307192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.308188 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.308192 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.308720 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.308910 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.309677 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.309689 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.312457 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.314315 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.314323 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.314327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.314666 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.315085 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.315086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.315086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.315237 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.315606 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.315614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.315836 (+ 222us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.316594 (+ 758us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.316811 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":39,"thread_start_us":124,"threads_started":1}
I20260619 12:05:03.317358 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.302919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:34663 (local address 127.0.203.190:35585)
0619 12:05:03.302965 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.302967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.303461 (+ 494us) server_negotiation.cc:407] Connection header received
0619 12:05:03.304722 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.304724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.304748 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.304814 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.305644 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.306438 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308302 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.308599 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308643 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.313349 (+ 4706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.313351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.313351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.313361 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.316046 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316537 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.316538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.316539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.316604 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316779 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.316780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.316781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.316924 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.316972 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.317153 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.317241 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:05:03.317459 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.303325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:51021 (local address 127.0.203.188:38511)
0619 12:05:03.303396 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.303398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.303562 (+ 164us) server_negotiation.cc:407] Connection header received
0619 12:05:03.304568 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.304570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.304584 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.304635 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.305613 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.306432 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308435 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.308744 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308785 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.314604 (+ 5819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.314606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.314606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.314616 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.316211 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316668 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.316669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.316669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.316727 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316917 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.316918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.316918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.317047 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.317092 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.317272 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.317365 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:05:03.317807 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.303239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.203.130:51021)
0619 12:05:03.303474 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.303485 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.304420 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.304737 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.304738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.305197 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.305495 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.305502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.306598 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.306599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.308314 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.308317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.308863 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.308922 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.309775 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.309779 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.313210 (+ 3431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.316364 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.316365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.316586 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.316788 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316788 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.316789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.316841 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.317123 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.317124 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.317167 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.317582 (+ 415us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.317696 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":31,"thread_start_us":69,"threads_started":1}
I20260619 12:05:03.318235 8024 heartbeater.cc:347] Connected to a master server at 127.0.203.189:34321
I20260619 12:05:03.318370 8024 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.318570 8022 heartbeater.cc:347] Connected to a master server at 127.0.203.188:38511
I20260619 12:05:03.318629 8022 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.318730 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.302876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:34663)
0619 12:05:03.303384 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.303396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.304390 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.304837 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.304838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.305177 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.305495 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.305502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.306598 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.306599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.308178 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.308182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.308735 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.308736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.308908 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.309692 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.309698 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.312623 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.316176 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316178 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.316178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.316448 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.316665 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.316665 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.316666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.316720 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.317004 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.317005 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.317053 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.317793 (+ 740us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.318597 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":103,"thread_start_us":121,"threads_started":1}
I20260619 12:05:03.318848 8022 heartbeater.cc:511] Master 127.0.203.188:38511 requested a full tablet report, sending...
I20260619 12:05:03.319128 8024 heartbeater.cc:511] Master 127.0.203.189:34321 requested a full tablet report, sending...
I20260619 12:05:03.319141 8023 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35585
I20260619 12:05:03.319208 8023 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.319271 8023 heartbeater.cc:511] Master 127.0.203.190:35585 requested a full tablet report, sending...
I20260619 12:05:03.319512 7706 ts_manager.cc:194] Registered new tserver with Master: efbd66f920e54163be6877706e7260a0 (127.0.203.130:34055)
I20260619 12:05:03.319538 7520 ts_manager.cc:194] Registered new tserver with Master: efbd66f920e54163be6877706e7260a0 (127.0.203.130:34055)
I20260619 12:05:03.319893 6923 ts_manager.cc:194] Re-registered known tserver with Master: efbd66f920e54163be6877706e7260a0 (127.0.203.130:34055)
I20260619 12:05:03.320468 6923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:34663
W20260619 12:05:03.407640 8029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:03.407881 8029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:03.407908 8029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:03.409222 8029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:03.409277 8029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:03.409307 8029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:03.410948 8029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab
--principal=oryx/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:37085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=46191
--tserver_master_addrs=127.0.203.190:35585,127.0.203.189:34321,127.0.203.188:38511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:03.411233 8029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:03.411530 8029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:03.417311 8037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:03.417315 8035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:03.417323 8034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:03.417634 8029 server_base.cc:1034] running on GCE node
I20260619 12:05:03.417886 8029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:03.418160 8029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:03.419333 8029 hybrid_clock.cc:648] HybridClock initialized: now 1781870703419312 us; error 38 us; skew 500 ppm
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:03.422070 8029 init.cc:377] Logged in from keytab as oryx/127.0.203.131@KRBTEST.COM (short username oryx)
I20260619 12:05:03.422676 8029 webserver.cc:460] Webserver started at http://127.0.203.131:46191/ using document root <none> and password file <none>
I20260619 12:05:03.422984 8029 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:03.423067 8029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:03.424629 8029 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:03.425294 8044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.425521 8029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:05:03.425614 8029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "23961998e5c7406f97ea2e6453f501f3"
format_stamp: "Formatted at 2026-06-19 12:05:02 on dist-test-slave-44vj"
server_key: "79be76cfa2a6c2454b2de3dba111044f"
server_key_iv: "a28bb682c8502857eba1ff625d3b63f7"
server_key_version: "encryptionkey@0"
I20260619 12:05:03.425987 8029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:03.457082 8029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:03.459245 8029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:03.459432 8029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:03.459671 8029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:03.459993 8029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:03.460035 8029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.460058 8029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:03.460085 8029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:03.466902 8029 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:37085
I20260619 12:05:03.467015 8157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:37085 every 8 connection(s)
I20260619 12:05:03.467411 8029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:05:03.474871 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 8029
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.188@KRBTEST.COM
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
Jun 19 12:05:03 dist-test-slave-44vj krb5kdc[6878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870703, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.189@KRBTEST.COM
I20260619 12:05:03.482743 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.468312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:60021 (local address 127.0.203.189:34321)
0619 12:05:03.468453 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.468455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.468783 (+ 328us) server_negotiation.cc:407] Connection header received
0619 12:05:03.470029 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.470030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.470042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.470101 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.471461 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.471979 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.473483 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.473855 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.473931 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.478380 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.478382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.478382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.478390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.481020 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.481798 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.481799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.481800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.481877 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.482064 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.482064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.482065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.482214 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.482269 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.482535 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.482642 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 12:05:03.482939 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.468775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.203.131:54265)
0619 12:05:03.468956 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.468967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.469829 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.470258 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.470259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.470764 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.471090 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.471097 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.472149 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.472150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.473380 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.473383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.474404 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.474405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.474466 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.475114 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.475119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.477195 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.479822 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.479832 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.479846 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.480186 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.481508 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.481508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.481509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.481700 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.482021 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.482029 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.482231 (+ 202us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.482343 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.482602 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":128,"thread_start_us":117,"threads_started":1}
I20260619 12:05:03.483939 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.468295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.203.131:60021)
0619 12:05:03.468619 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.468665 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.469811 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.470130 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.470135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.470831 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.471091 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.471100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.472129 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.472130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.473383 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.473385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.473953 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.473954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.474131 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.474964 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.474974 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.478288 (+ 3314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.481123 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.481125 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.481125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.481713 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.481942 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.481943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.481943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.482005 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.482312 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.482313 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.482364 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.482427 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.483809 (+ 1382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":1372,"thread_start_us":229,"threads_started":1}
I20260619 12:05:03.484126 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.468298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:48663)
0619 12:05:03.468615 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.468663 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.469817 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.470131 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.470135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.470835 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:03.471091 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.471100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.472199 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.472200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.474123 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.474125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.474641 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.474642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.474728 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.475325 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:03.475330 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:03.478604 (+ 3274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:03.481111 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.481114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.481114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.481519 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.482739 (+ 1220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:03.482740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:03.482740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:03.482797 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:03.483063 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:03.483065 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:03.483563 (+ 498us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.483650 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.483992 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":1062,"thread_start_us":236,"threads_started":1}
I20260619 12:05:03.484400 8160 heartbeater.cc:347] Connected to a master server at 127.0.203.189:34321
I20260619 12:05:03.484495 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.468389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:48663 (local address 127.0.203.190:35585)
0619 12:05:03.468451 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.468453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.468770 (+ 317us) server_negotiation.cc:407] Connection header received
0619 12:05:03.469950 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.469951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.469965 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.470032 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.471235 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.472072 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.474241 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.474554 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.474640 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.478833 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.478835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.478835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.478843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.480970 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.482597 (+ 1627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.482598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.482599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.482665 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.482858 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.482859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.482859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.482980 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.483039 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.484323 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.484408 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:05:03.484519 8160 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.484764 8160 heartbeater.cc:511] Master 127.0.203.189:34321 requested a full tablet report, sending...
I20260619 12:05:03.485198 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.468968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:54265 (local address 127.0.203.188:38511)
0619 12:05:03.469035 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.469037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.469048 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:03.470159 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.470160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.470181 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.470227 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.471207 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.472072 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.474040 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.474326 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.474389 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.477327 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.477328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.477329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.477338 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.479297 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.481347 (+ 2050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.481348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.481349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.481410 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.481784 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.481785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.481785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.481935 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.481995 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.485005 (+ 3010us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.485109 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:05:03.485275 7520 ts_manager.cc:194] Registered new tserver with Master: 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:03.485283 8159 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35585
I20260619 12:05:03.485340 8159 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.485426 8159 heartbeater.cc:511] Master 127.0.203.190:35585 requested a full tablet report, sending...
I20260619 12:05:03.485497 8158 heartbeater.cc:347] Connected to a master server at 127.0.203.188:38511
I20260619 12:05:03.485540 8158 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:03.485615 8158 heartbeater.cc:511] Master 127.0.203.188:38511 requested a full tablet report, sending...
I20260619 12:05:03.485708 6923 ts_manager.cc:194] Re-registered known tserver with Master: 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:03.485924 7706 ts_manager.cc:194] Registered new tserver with Master: 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:03.486160 6923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:48663
I20260619 12:05:03.487865 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.476303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.0.203.189:34321)
0619 12:05:03.476452 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.476454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.476464 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:03.476489 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.476490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.476504 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.476590 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.477382 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.478080 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.479089 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.479423 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.479489 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.482516 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.482518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.482518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.482524 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.486370 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.486819 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.486820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.486821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.486899 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.487121 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.487122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.487122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.487263 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.487628 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.487675 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.487780 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260619 12:05:03.488886 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.477866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.0.203.190:35585)
0619 12:05:03.478084 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.478086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.479750 (+ 1664us) server_negotiation.cc:407] Connection header received
0619 12:05:03.479775 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.479776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.479802 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.481636 (+ 1834us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.481680 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.482432 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.484161 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.484433 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.484754 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.485796 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.485797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.485797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.485803 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.487708 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.488106 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.488108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.488108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.488172 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.488371 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.488372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.488372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.488501 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.488604 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.488709 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.488788 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":110,"threads_started":1}
I20260619 12:05:03.490074 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.479013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.0.203.188:38511)
0619 12:05:03.479267 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.479268 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.479597 (+ 329us) server_negotiation.cc:407] Connection header received
0619 12:05:03.479683 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.479684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.479697 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.479740 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.480529 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.481330 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.482444 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.482719 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.482799 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.487115 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.487118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.487118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.487128 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.488881 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.489252 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.489254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.489254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.489325 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.489543 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.489543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.489543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.489678 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.489788 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.489891 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.489974 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":73,"threads_started":1}
I20260619 12:05:03.490559 6923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:05:03.492290 6923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:03.501040 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.496991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.0.203.130:34055)
0619 12:05:03.497186 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.497192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.497670 (+ 478us) server_negotiation.cc:407] Connection header received
0619 12:05:03.497711 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.497713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.497729 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.497814 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:03.498032 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.499008 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.500136 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.500667 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.500733 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.500824 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.500896 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.500940 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260619 12:05:03.501142 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.496740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:37085 (local address 127.0.0.1:60700)
0619 12:05:03.497015 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.497026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.497079 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.497322 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.497323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.497327 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:03.497409 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.497412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.498453 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.498454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.499878 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.499881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.500799 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.500800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.500880 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.500887 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.500917 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.500985 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":56,"threads_started":1}
I20260619 12:05:03.501086 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.496885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34055 (local address 127.0.0.1:44146)
0619 12:05:03.497259 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.497267 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.497306 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.497833 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.497834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.497844 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:03.497945 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.497947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.499089 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.499091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.500053 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.500057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.500753 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.500754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.500819 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.500825 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.500880 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.500940 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":22,"thread_start_us":72,"threads_started":1}
I20260619 12:05:03.501389 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.496952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.0.203.131:37085)
0619 12:05:03.497140 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.497145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.497158 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:05:03.497201 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.497204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.497220 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.497316 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:03.497485 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.498383 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.499957 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.500700 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.501130 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.501209 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.501239 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.501290 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":116,"threads_started":1}
I20260619 12:05:03.502322 8092 tablet_service.cc:1467] Processing CreateTablet for tablet a72a621cf18841cfbc86337b756a1f08 (DEFAULT_TABLE table=test-table [id=7a9f3fa27324458590a154ee70aff562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:03.502810 8092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72a621cf18841cfbc86337b756a1f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:03.502818 7956 tablet_service.cc:1467] Processing CreateTablet for tablet a72a621cf18841cfbc86337b756a1f08 (DEFAULT_TABLE table=test-table [id=7a9f3fa27324458590a154ee70aff562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:03.503291 7956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72a621cf18841cfbc86337b756a1f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:03.507329 8186 tablet_bootstrap.cc:492] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Bootstrap starting.
I20260619 12:05:03.507488 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.496746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:42535 (local address 127.0.0.1:34668)
0619 12:05:03.497144 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.497151 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.497184 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.497591 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.497592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.497595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:03.497666 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.497669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.498906 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.498907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.500086 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.500088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.501099 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.501100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.501398 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.501403 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.501507 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.507320 (+ 5813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":37,"thread_start_us":50,"threads_started":1}
I20260619 12:05:03.507820 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.496850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.0.203.129:42535)
0619 12:05:03.497431 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.497435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.497446 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:03.497489 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.497492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.497510 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.497595 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:03.497743 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.498807 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.500183 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.500924 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.500984 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.501066 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.507647 (+ 6581us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.507723 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":114,"threads_started":1}
I20260619 12:05:03.508217 8186 tablet_bootstrap.cc:654] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:03.508531 8186 log.cc:824] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:03.508656 7819 tablet_service.cc:1467] Processing CreateTablet for tablet a72a621cf18841cfbc86337b756a1f08 (DEFAULT_TABLE table=test-table [id=7a9f3fa27324458590a154ee70aff562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:03.509042 7819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a72a621cf18841cfbc86337b756a1f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:03.509137 8185 tablet_bootstrap.cc:492] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Bootstrap starting.
I20260619 12:05:03.509518 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.502443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.0.203.188:38511)
0619 12:05:03.502506 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.502508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.502632 (+ 124us) server_negotiation.cc:407] Connection header received
0619 12:05:03.502762 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.502767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.502794 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.502838 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:03.503034 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.503683 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.508413 (+ 4730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.509019 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.509099 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.509177 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.509381 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.509442 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:05:03.509729 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.502304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.0.1:38292)
0619 12:05:03.502574 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.502586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.502672 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.502905 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.502906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.502907 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:03.502962 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.502963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.503759 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.503759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.508296 (+ 4537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.508301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.509220 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.509221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.509286 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.509290 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.509599 (+ 309us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.509664 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250}
I20260619 12:05:03.510166 8185 tablet_bootstrap.cc:654] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:03.511329 8186 tablet_bootstrap.cc:492] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: No bootstrap required, opened a new log
I20260619 12:05:03.511487 8186 ts_tablet_manager.cc:1397] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260619 12:05:03.511999 7716 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 12:05:03.513342 7745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5843ca67bb6f4ea1ad6ddc7965a21eaa. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:03.513485 7745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:05:03.514110 8186 raft_consensus.cc:348] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.514299 8186 raft_consensus.cc:374] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:03.514331 8186 raft_consensus.cc:729] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23961998e5c7406f97ea2e6453f501f3, State: Initialized, Role: FOLLOWER
I20260619 12:05:03.514492 8186 consensus_queue.cc:260] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.514954 8186 ts_tablet_manager.cc:1428] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:05:03.515043 8159 heartbeater.cc:503] Master 127.0.203.190:35585 was elected leader, sending a full tablet report...
I20260619 12:05:03.515342 8185 log.cc:824] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:03.517108 8185 tablet_bootstrap.cc:492] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: No bootstrap required, opened a new log
I20260619 12:05:03.517205 8185 ts_tablet_manager.cc:1397] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.002s
I20260619 12:05:03.518221 8194 tablet_bootstrap.cc:492] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Bootstrap starting.
I20260619 12:05:03.518972 8194 tablet_bootstrap.cc:654] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:03.519290 8194 log.cc:824] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:03.519526 8185 raft_consensus.cc:348] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.519724 8185 raft_consensus.cc:374] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:03.519758 8185 raft_consensus.cc:729] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbd66f920e54163be6877706e7260a0, State: Initialized, Role: FOLLOWER
I20260619 12:05:03.519915 8185 consensus_queue.cc:260] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.520187 8194 tablet_bootstrap.cc:492] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: No bootstrap required, opened a new log
I20260619 12:05:03.520226 8185 ts_tablet_manager.cc:1428] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 12:05:03.520262 8194 ts_tablet_manager.cc:1397] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:05:03.520352 8023 heartbeater.cc:503] Master 127.0.203.190:35585 was elected leader, sending a full tablet report...
I20260619 12:05:03.522213 8194 raft_consensus.cc:348] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.522375 8194 raft_consensus.cc:374] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:03.522398 8194 raft_consensus.cc:729] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dfb4466a7ac441baa3bfd343cbf8704, State: Initialized, Role: FOLLOWER
I20260619 12:05:03.522502 8194 consensus_queue.cc:260] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.522753 8194 ts_tablet_manager.cc:1428] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:05:03.522858 7886 heartbeater.cc:503] Master 127.0.203.190:35585 was elected leader, sending a full tablet report...
W20260619 12:05:03.553757 8025 tablet.cc:2369] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:05:03.648082 7888 tablet.cc:2369] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:05:03.718376 8161 tablet.cc:2369] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:05:03.752907 8196 raft_consensus.cc:482] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:03.753101 8196 raft_consensus.cc:504] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.753602 8196 leader_election.cc:288] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129:42535), 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:03.756728 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.754006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:34207 (local address 127.0.203.129:42535)
0619 12:05:03.754090 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.754093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.754128 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:05:03.754164 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.754165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.754176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.754226 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:03.754398 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.754937 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.755994 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.756380 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.756445 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.756458 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.756593 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.756635 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:05:03.756728 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.753869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:42535 (local address 127.0.203.130:34207)
0619 12:05:03.753970 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.753988 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.754087 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.754237 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.754239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.754240 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:03.754331 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.754334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.755023 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.755024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.755906 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.755909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.756456 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.756457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.756533 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.756537 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.756582 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.756640 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 12:05:03.756795 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.754009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:48429 (local address 127.0.203.131:37085)
0619 12:05:03.754123 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.754126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.754143 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:05:03.754242 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.754244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.754256 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.754316 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:03.754492 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.754991 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.756061 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.756428 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.756504 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.756516 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.756657 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.756700 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 12:05:03.756852 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.753869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:37085 (local address 127.0.203.130:48429)
0619 12:05:03.754043 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:03.754082 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:03.754160 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:03.754327 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:03.754328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:03.754329 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:03.754419 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.754421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.755095 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.755096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.755964 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:03.755967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.756512 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.756513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:03.756588 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.756592 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:03.756646 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:05:03.756716 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":111}
I20260619 12:05:03.757310 7839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72a621cf18841cfbc86337b756a1f08" candidate_uuid: "efbd66f920e54163be6877706e7260a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" is_pre_election: true
I20260619 12:05:03.757328 8112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72a621cf18841cfbc86337b756a1f08" candidate_uuid: "efbd66f920e54163be6877706e7260a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23961998e5c7406f97ea2e6453f501f3" is_pre_election: true
I20260619 12:05:03.757496 7839 raft_consensus.cc:2454] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efbd66f920e54163be6877706e7260a0 in term 0.
I20260619 12:05:03.757498 8112 raft_consensus.cc:2454] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efbd66f920e54163be6877706e7260a0 in term 0.
I20260619 12:05:03.757789 7910 leader_election.cc:302] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23961998e5c7406f97ea2e6453f501f3, efbd66f920e54163be6877706e7260a0; no voters:
I20260619 12:05:03.758006 8196 raft_consensus.cc:2781] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:03.758065 8196 raft_consensus.cc:482] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:03.758085 8196 raft_consensus.cc:3037] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:03.758874 8196 raft_consensus.cc:504] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.759037 8196 leader_election.cc:288] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [CANDIDATE]: Term 1 election: Requested vote from peers 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129:42535), 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:03.759307 7839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72a621cf18841cfbc86337b756a1f08" candidate_uuid: "efbd66f920e54163be6877706e7260a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dfb4466a7ac441baa3bfd343cbf8704"
I20260619 12:05:03.759280 8112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a72a621cf18841cfbc86337b756a1f08" candidate_uuid: "efbd66f920e54163be6877706e7260a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23961998e5c7406f97ea2e6453f501f3"
I20260619 12:05:03.759435 7839 raft_consensus.cc:3037] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:03.759485 8112 raft_consensus.cc:3037] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:03.760501 7839 raft_consensus.cc:2454] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efbd66f920e54163be6877706e7260a0 in term 1.
I20260619 12:05:03.760501 8112 raft_consensus.cc:2454] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efbd66f920e54163be6877706e7260a0 in term 1.
I20260619 12:05:03.760735 7910 leader_election.cc:302] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23961998e5c7406f97ea2e6453f501f3, efbd66f920e54163be6877706e7260a0; no voters:
I20260619 12:05:03.760951 8196 raft_consensus.cc:2781] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:03.761134 8196 raft_consensus.cc:686] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 1 LEADER]: Becoming Leader. State: Replica: efbd66f920e54163be6877706e7260a0, State: Running, Role: LEADER
I20260619 12:05:03.761247 8196 consensus_queue.cc:237] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } }
I20260619 12:05:03.762274 6922 catalog_manager.cc:5495] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to efbd66f920e54163be6877706e7260a0 (127.0.203.130). New cstate: current_term: 1 leader_uuid: "efbd66f920e54163be6877706e7260a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } health_report { overall_health: HEALTHY } } }
I20260619 12:05:03.763056 7481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: NON_VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: true } } }
I20260619 12:05:03.763172 7481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:03.764107 7670 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: attempting to promote NON_VOTER 622b770926624591a1afcea86866d2e9 to VOTER
I20260619 12:05:03.764261 7670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:03.764819 7716 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 LEARNER]: Refusing update from remote peer 5843ca67bb6f4ea1ad6ddc7965a21eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 12:05:03.764853 7529 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Refusing update from remote peer 5843ca67bb6f4ea1ad6ddc7965a21eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 12:05:03.765075 7481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260619 12:05:03.765189 7670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260619 12:05:03.765879 8201 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 622b770926624591a1afcea86866d2e9 (127.0.203.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.766047 7716 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 622b770926624591a1afcea86866d2e9 (127.0.203.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.766192 7529 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 622b770926624591a1afcea86866d2e9 (127.0.203.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.766942 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.767026 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:03.767081 7670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.767143 7670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:03.767263 7745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.767295 7670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.767383 7670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:03.767385 7745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:05:03.767732 8203 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:03.767831 8203 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:05:03.791302 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.788584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.0.203.130:34055)
0619 12:05:03.788669 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.788671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.788682 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:03.788736 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.788737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.788756 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.788830 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:03.789055 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.789553 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.790445 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.790653 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.790728 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.790905 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:03.791018 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:03.791085 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.791175 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.791196 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:05:03.793434 8112 raft_consensus.cc:1261] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Refusing update from remote peer efbd66f920e54163be6877706e7260a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:03.793488 7839 raft_consensus.cc:1261] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 FOLLOWER]: Refusing update from remote peer efbd66f920e54163be6877706e7260a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:03.793819 8196 consensus_queue.cc:1035] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:03.794009 8198 consensus_queue.cc:1035] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:03.798158 8205 mvcc.cc:205] Tried to move back new op lower bound from 7298542402734534656 to 7298542402606751744. Current Snapshot: MvccSnapshot[applied={T|T < 7298542402734534656}]
I20260619 12:05:03.798287 8206 mvcc.cc:205] Tried to move back new op lower bound from 7298542402734534656 to 7298542402606751744. Current Snapshot: MvccSnapshot[applied={T|T < 7298542402734534656}]
I20260619 12:05:03.800395 8207 mvcc.cc:205] Tried to move back new op lower bound from 7298542402734534656 to 7298542402606751744. Current Snapshot: MvccSnapshot[applied={T|T < 7298542402734534656 or (T in {7298542402734534656})}]
I20260619 12:05:03.802110 6922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:03.802249 6922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:03.803949 6922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5843ca67bb6f4ea1ad6ddc7965a21eaa: Sending DeleteTablet for 3 replicas of tablet a72a621cf18841cfbc86337b756a1f08
I20260619 12:05:03.804445 7956 tablet_service.cc:1514] Processing DeleteTablet for tablet a72a621cf18841cfbc86337b756a1f08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:03 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:44146
I20260619 12:05:03.804468 7819 tablet_service.cc:1514] Processing DeleteTablet for tablet a72a621cf18841cfbc86337b756a1f08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:03 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:34668
I20260619 12:05:03.804687 8216 tablet_replica.cc:330] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: stopping tablet replica
I20260619 12:05:03.804800 8216 raft_consensus.cc:2229] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:05:03.804847 8092 tablet_service.cc:1514] Processing DeleteTablet for tablet a72a621cf18841cfbc86337b756a1f08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:03 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:60700
I20260619 12:05:03.804956 8216 raft_consensus.cc:2258] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:03.805110 8215 tablet_replica.cc:330] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: stopping tablet replica
I20260619 12:05:03.805199 8217 tablet_replica.cc:330] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: stopping tablet replica
I20260619 12:05:03.805243 8215 raft_consensus.cc:2229] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:03.805312 8217 raft_consensus.cc:2229] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:03.805373 8215 raft_consensus.cc:2258] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:03.805413 8217 raft_consensus.cc:2258] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:03.805855 8217 ts_tablet_manager.cc:1905] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:03.806114 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 6891
I20260619 12:05:03.808373 8217 ts_tablet_manager.cc:1918] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:03.808449 8217 log.cc:1196] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/a72a621cf18841cfbc86337b756a1f08
I20260619 12:05:03.808655 8217 ts_tablet_manager.cc:1939] T a72a621cf18841cfbc86337b756a1f08 P 23961998e5c7406f97ea2e6453f501f3: Deleting consensus metadata
I20260619 12:05:03.811668 8216 ts_tablet_manager.cc:1905] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:05:03.811869 8046 connection.cc:535] server connection from 127.0.0.1:60700 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:05:03.812573 8215 ts_tablet_manager.cc:1905] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:03.813866 8216 ts_tablet_manager.cc:1918] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:03.813978 8216 log.cc:1196] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/a72a621cf18841cfbc86337b756a1f08
I20260619 12:05:03.814253 8216 ts_tablet_manager.cc:1939] T a72a621cf18841cfbc86337b756a1f08 P efbd66f920e54163be6877706e7260a0: Deleting consensus metadata
W20260619 12:05:03.814666 7910 connection.cc:441] server connection from 127.0.0.1:44146 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44146 (request call id 1) could send its response
I20260619 12:05:03.814778 8215 ts_tablet_manager.cc:1918] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:03.814867 8215 log.cc:1196] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/a72a621cf18841cfbc86337b756a1f08
I20260619 12:05:03.815141 8215 ts_tablet_manager.cc:1939] T a72a621cf18841cfbc86337b756a1f08 P 0dfb4466a7ac441baa3bfd343cbf8704: Deleting consensus metadata
W20260619 12:05:03.815629 7775 connection.cc:441] server connection from 127.0.0.1:34668 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34668 (request call id 1) could send its response
I20260619 12:05:03.822742 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.814867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38302 (local address 127.0.203.188:38511)
0619 12:05:03.814932 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.814935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.815082 (+ 147us) server_negotiation.cc:407] Connection header received
0619 12:05:03.815195 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.815196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.815208 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.815237 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.816081 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.816576 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.817434 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.817740 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.817778 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.819557 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.819558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.819558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.819566 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.821197 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.821833 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.821835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.821835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.821896 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.822104 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.822104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.822105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.822249 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.822329 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.822542 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.822620 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:05:03.824239 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.815190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.0.203.189:34321)
0619 12:05:03.815242 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.815244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.815766 (+ 522us) server_negotiation.cc:407] Connection header received
0619 12:05:03.815937 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.815939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.815960 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.816047 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.817172 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.817857 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.818684 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.818981 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.819024 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.821069 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.821070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.821070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.821077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.822774 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.823540 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.823541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.823541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.823599 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.823779 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.823779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.823780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.823887 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.823971 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.824068 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.824145 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:05:03.825342 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:03.805488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.0.203.188:38511)
0619 12:05:03.805739 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:03.805741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:03.815537 (+ 9796us) server_negotiation.cc:407] Connection header received
0619 12:05:03.817272 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:03.817274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:03.817287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:03.817317 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:03.818053 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.819256 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.820263 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:03.820586 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:03.820660 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:03.822327 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:03.822329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:03.822329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:03.822336 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:03.824067 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.824503 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.824505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.824505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.824559 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:03.824774 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:03.824775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:03.824775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:03.824920 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:03.825025 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:03.825167 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 12:05:03.825256 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260619 12:05:04.799417 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:04.798959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:42031)
0619 12:05:04.799197 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:04.799316 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":90,"threads_started":1}
W20260619 12:05:04.799873 8023 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:35585 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:04.800209 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:04.799962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:54797)
0619 12:05:04.800095 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:04.800162 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
W20260619 12:05:04.800491 8159 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:35585 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:04.800874 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:04.800626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:37639)
0619 12:05:04.800766 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:04.800838 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
W20260619 12:05:04.801110 7886 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:35585 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:05.304643 8228 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5843ca67bb6f4ea1ad6ddc7965a21eaa)
I20260619 12:05:05.304827 8228 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:05.304822 8229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5843ca67bb6f4ea1ad6ddc7965a21eaa)
I20260619 12:05:05.305009 8229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:05.305100 8228 leader_election.cc:288] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585), 622b770926624591a1afcea86866d2e9 (127.0.203.188:38511)
I20260619 12:05:05.305352 8229 leader_election.cc:288] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585), a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189:34321)
I20260619 12:05:05.305670 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.305270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59284)
0619 12:05:05.305558 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.305625 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":45,"threads_started":1}
W20260619 12:05:05.305827 7506 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:05.306011 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.305559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59294)
0619 12:05:05.305867 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.305923 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":77,"threads_started":1}
W20260619 12:05:05.306188 7692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:05.306571 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.306452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59310)
0619 12:05:05.306509 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.306545 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:05:05.306668 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.306540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59314)
0619 12:05:05.306605 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.306640 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 12:05:05.306775 7692 leader_election.cc:334] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585): Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
W20260619 12:05:05.307402 7506 leader_election.cc:334] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585): Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:05.309491 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.305613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.0.203.188:38511)
0619 12:05:05.305700 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:05.305702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:05.305916 (+ 214us) server_negotiation.cc:407] Connection header received
0619 12:05:05.305946 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:05.305947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:05.305959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:05.306029 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:05.306253 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.307002 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.308488 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.309124 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.309214 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:05.309229 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:05.309367 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 12:05:05.309413 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64,"thread_start_us":38,"threads_started":1}
I20260619 12:05:05.309495 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.305408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:38511 (local address 127.0.0.1:38310)
0619 12:05:05.305762 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.305775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:05.305859 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:05.306088 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:05.306090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:05.306093 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:05.306170 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:05.306173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.307089 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.307090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:05.308373 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:05.308376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.309231 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.309232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:05.309291 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:05.309298 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:05:05.309351 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:05:05.309414 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":54,"threads_started":1}
I20260619 12:05:05.309659 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.305496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34321 (local address 127.0.0.1:59150)
0619 12:05:05.305846 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.305860 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:05.305983 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:05.306303 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:05.306304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:05.306309 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:05.306381 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:05.306385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.307315 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.307316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:05.308528 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:05.308531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.309304 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.309305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:05.309364 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:05.309368 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:05.309418 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:05:05.309500 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":45,"threads_started":1}
I20260619 12:05:05.309844 7716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5688560fd604d358b3f13b9cc8e3724" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "622b770926624591a1afcea86866d2e9" is_pre_election: true
I20260619 12:05:05.309932 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.305644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.0.203.189:34321)
0619 12:05:05.306134 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:05.306136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:05.306146 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:05.306162 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:05.306163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:05.306173 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:05.306250 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:05.306661 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.307231 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.308620 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:05.309223 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:05.309291 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:05.309301 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:05.309433 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 12:05:05.309489 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":52,"threads_started":1}
I20260619 12:05:05.309989 7716 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5688560fd604d358b3f13b9cc8e3724 in term 1.
I20260619 12:05:05.310143 7506 leader_election.cc:302] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 622b770926624591a1afcea86866d2e9, a5688560fd604d358b3f13b9cc8e3724; no voters: 5843ca67bb6f4ea1ad6ddc7965a21eaa
I20260619 12:05:05.310127 7529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "622b770926624591a1afcea86866d2e9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5688560fd604d358b3f13b9cc8e3724" is_pre_election: true
I20260619 12:05:05.310248 7529 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 622b770926624591a1afcea86866d2e9 in term 1.
I20260619 12:05:05.310321 8228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 12:05:05.310362 8228 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5843ca67bb6f4ea1ad6ddc7965a21eaa)
I20260619 12:05:05.310381 8228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:05:05.310467 7691 leader_election.cc:302] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 622b770926624591a1afcea86866d2e9, a5688560fd604d358b3f13b9cc8e3724; no voters: 5843ca67bb6f4ea1ad6ddc7965a21eaa
I20260619 12:05:05.310572 8229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 12:05:05.310639 8229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5843ca67bb6f4ea1ad6ddc7965a21eaa)
I20260619 12:05:05.310662 8229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:05:05.311305 8228 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:05.311487 8228 leader_election.cc:288] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [CANDIDATE]: Term 2 election: Requested vote from peers 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585), 622b770926624591a1afcea86866d2e9 (127.0.203.188:38511)
I20260619 12:05:05.311460 8229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:05.311609 8229 leader_election.cc:288] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 2 election: Requested vote from peers 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585), a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189:34321)
I20260619 12:05:05.311713 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.311597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59326)
0619 12:05:05.311649 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.311687 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 12:05:05.311759 7716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5688560fd604d358b3f13b9cc8e3724" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "622b770926624591a1afcea86866d2e9"
I20260619 12:05:05.311784 7529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "622b770926624591a1afcea86866d2e9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5688560fd604d358b3f13b9cc8e3724"
I20260619 12:05:05.311830 7716 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a5688560fd604d358b3f13b9cc8e3724 in current term 2: Already voted for candidate 622b770926624591a1afcea86866d2e9 in this term.
I20260619 12:05:05.311861 7529 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 622b770926624591a1afcea86866d2e9 in current term 2: Already voted for candidate a5688560fd604d358b3f13b9cc8e3724 in this term.
I20260619 12:05:05.312189 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.311732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59338)
0619 12:05:05.312126 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.312160 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374}
I20260619 12:05:05.312222 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.312127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59344)
0619 12:05:05.312179 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.312204 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:05:05.312500 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.312431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:59358)
0619 12:05:05.312467 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.312487 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 12:05:05.312494 7506 leader_election.cc:334] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585): Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:05.312538 7506 leader_election.cc:302] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5688560fd604d358b3f13b9cc8e3724; no voters: 5843ca67bb6f4ea1ad6ddc7965a21eaa, 622b770926624591a1afcea86866d2e9
W20260619 12:05:05.312572 7692 leader_election.cc:334] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585): Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:05.312610 8228 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 12:05:05.312620 7692 leader_election.cc:302] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 622b770926624591a1afcea86866d2e9; no voters: 5843ca67bb6f4ea1ad6ddc7965a21eaa, a5688560fd604d358b3f13b9cc8e3724
I20260619 12:05:05.312680 8229 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 12:05:05.800873 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.800452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:50485)
0619 12:05:05.800724 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.800799 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":93,"threads_started":1}
I20260619 12:05:05.801316 8239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.800842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:40751)
0619 12:05:05.801122 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.801165 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":57,"threads_started":1}
I20260619 12:05:05.801919 8240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:05.801495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:56797)
0619 12:05:05.801838 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:05.801876 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":65,"threads_started":1}
I20260619 12:05:06.802145 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.801886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:40895)
0619 12:05:06.802014 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.802065 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":54,"threads_started":1}
I20260619 12:05:06.802400 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.801725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:49253)
0619 12:05:06.802198 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.802332 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":89,"threads_started":1}
I20260619 12:05:06.802862 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.802384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:49055)
0619 12:05:06.802762 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.802810 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":74,"threads_started":1}
I20260619 12:05:06.866262 8244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:06.866379 8244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:06.866564 8244 leader_election.cc:288] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585), a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189:34321)
I20260619 12:05:06.866905 7529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "622b770926624591a1afcea86866d2e9" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5688560fd604d358b3f13b9cc8e3724" is_pre_election: true
I20260619 12:05:06.867002 7529 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 622b770926624591a1afcea86866d2e9 in term 2.
I20260619 12:05:06.867023 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.866771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:60566)
0619 12:05:06.866925 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.866965 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260619 12:05:06.867219 7691 leader_election.cc:302] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 622b770926624591a1afcea86866d2e9, a5688560fd604d358b3f13b9cc8e3724; no voters:
I20260619 12:05:06.867342 8244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 12:05:06.867427 8244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:06.867470 8244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:05:06.867544 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.867396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:60578)
0619 12:05:06.867449 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.867505 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:05:06.867664 7692 leader_election.cc:334] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585): Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:06.868466 8244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:06.868615 8244 leader_election.cc:288] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 3 election: Requested vote from peers 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585), a5688560fd604d358b3f13b9cc8e3724 (127.0.203.189:34321)
I20260619 12:05:06.868816 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.868719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:60592)
0619 12:05:06.868756 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.868787 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 12:05:06.868871 7529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "622b770926624591a1afcea86866d2e9" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a5688560fd604d358b3f13b9cc8e3724"
I20260619 12:05:06.868971 7529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:05:06.869215 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.869098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:60608)
0619 12:05:06.869165 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.869194 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 12:05:06.869282 7692 leader_election.cc:334] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585): Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
I20260619 12:05:06.869743 7529 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 622b770926624591a1afcea86866d2e9 in term 3.
I20260619 12:05:06.869938 7691 leader_election.cc:302] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 622b770926624591a1afcea86866d2e9, a5688560fd604d358b3f13b9cc8e3724; no voters: 5843ca67bb6f4ea1ad6ddc7965a21eaa
I20260619 12:05:06.870100 8244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:05:06.870267 8244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [term 3 LEADER]: Becoming Leader. State: Replica: 622b770926624591a1afcea86866d2e9, State: Running, Role: LEADER
I20260619 12:05:06.870353 8244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } }
I20260619 12:05:06.870769 8247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 622b770926624591a1afcea86866d2e9. Latest consensus state: current_term: 3 leader_uuid: "622b770926624591a1afcea86866d2e9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:06.870911 8247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:06.871158 8249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:06.871744 8249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:06.871829 8249 catalog_manager.cc:1259] Loaded cluster ID: d887bc3a8d6d40df98ced7921a80c4f8
I20260619 12:05:06.871860 8249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:05:06.872052 8249 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:06.872149 8249 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Loaded TSK: 0
I20260619 12:05:06.872327 8249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:05:06.928673 7706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:05:06.930459 7706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:06.931711 7529 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [term 3 FOLLOWER]: Refusing update from remote peer 622b770926624591a1afcea86866d2e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 12:05:06.931718 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.931585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:60626)
0619 12:05:06.931634 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.931666 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 12:05:06.931993 8247 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 12:05:06.932094 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.932013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.0.1:60630)
0619 12:05:06.932050 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.932076 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 12:05:06.932170 7692 consensus_peers.cc:487] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 -> Peer 5843ca67bb6f4ea1ad6ddc7965a21eaa (127.0.203.190:35585): Couldn't send request to peer 5843ca67bb6f4ea1ad6ddc7965a21eaa. Status: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 12:05:06.932677 8251 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 622b770926624591a1afcea86866d2e9. Latest consensus state: current_term: 3 leader_uuid: "622b770926624591a1afcea86866d2e9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:06.932814 8251 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:05:06.933465 8247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "622b770926624591a1afcea86866d2e9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:06.933571 8247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:06.933640 8251 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "622b770926624591a1afcea86866d2e9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5843ca67bb6f4ea1ad6ddc7965a21eaa" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35585 } } peers { permanent_uuid: "a5688560fd604d358b3f13b9cc8e3724" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34321 } attrs { promote: false } } peers { permanent_uuid: "622b770926624591a1afcea86866d2e9" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 38511 } attrs { promote: false } } }
I20260619 12:05:06.933715 8251 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5688560fd604d358b3f13b9cc8e3724 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:05:06.939092 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.935680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:42535 (local address 127.0.0.1:36770)
0619 12:05:06.936036 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.936043 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:06.936076 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:06.936458 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:06.936459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:06.936461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:06.936487 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:06.936488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.937166 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.937167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:06.938059 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:06.938061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.938848 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.938848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:06.938899 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:06.938903 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:06.938951 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:05:06.938998 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260619 12:05:06.939116 8259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.935886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.0.203.129:42535)
0619 12:05:06.936331 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:06.936334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:06.936343 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:05:06.936373 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:06.936374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:06.936385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:06.936456 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:06.936563 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.937076 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.938163 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.938762 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.938841 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:06.938857 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:06.938977 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 12:05:06.939027 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":75,"threads_started":1}
I20260619 12:05:06.939307 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.935577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34055 (local address 127.0.0.1:34306)
0619 12:05:06.935632 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.935645 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:06.935712 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:06.935983 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:06.935984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:06.935986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:06.936063 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:06.936065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.937060 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.937060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:06.938304 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:06.938307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.939056 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.939057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:06.939109 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:06.939112 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:05:06.939163 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:05:06.939241 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":1}
I20260619 12:05:06.939564 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.935665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.0.203.130:34055)
0619 12:05:06.935820 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:06.935822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:06.935830 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:05:06.935872 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:06.935873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:06.935886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:06.936125 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:06.936169 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.936970 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.938397 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.938972 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.939039 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:06.939051 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:06.939408 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 12:05:06.939493 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260619 12:05:06.939831 7956 tablet_service.cc:1467] Processing CreateTablet for tablet 6fef0234f21c47da91192fbd35447eb2 (DEFAULT_TABLE table=test-table [id=83c10cfdbec94c1d9d9098de2fcd66b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:06.939942 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.935639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:37085 (local address 127.0.0.1:51878)
0619 12:05:06.935896 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.935908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:06.935965 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:06.936240 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:06.936241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:06.936243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:06.936307 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:06.936309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.937273 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.937274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:06.938388 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:06.938389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.939695 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.939700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:06.939768 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:06.939772 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:06.939827 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:05:06.939868 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":127,"threads_started":1}
I20260619 12:05:06.940024 7956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fef0234f21c47da91192fbd35447eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:06.939990 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.935878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51878 (local address 127.0.203.131:37085)
0619 12:05:06.936099 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:06.936101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:06.936109 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:05:06.936136 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:06.936137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:06.936148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:06.936233 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:06.936507 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.937193 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.939089 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:06.939616 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:06.939692 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:06.939710 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:06.939850 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 12:05:06.939899 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":91,"threads_started":1}
I20260619 12:05:06.940161 7819 tablet_service.cc:1467] Processing CreateTablet for tablet 6fef0234f21c47da91192fbd35447eb2 (DEFAULT_TABLE table=test-table [id=83c10cfdbec94c1d9d9098de2fcd66b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:06.940326 7819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fef0234f21c47da91192fbd35447eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:06.940366 8092 tablet_service.cc:1467] Processing CreateTablet for tablet 6fef0234f21c47da91192fbd35447eb2 (DEFAULT_TABLE table=test-table [id=83c10cfdbec94c1d9d9098de2fcd66b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:06.940482 8092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fef0234f21c47da91192fbd35447eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:06.941466 8261 tablet_bootstrap.cc:492] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704: Bootstrap starting.
I20260619 12:05:06.941803 8262 tablet_bootstrap.cc:492] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: Bootstrap starting.
I20260619 12:05:06.941828 8261 tablet_bootstrap.cc:654] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:06.942189 8262 tablet_bootstrap.cc:654] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:06.942561 8263 tablet_bootstrap.cc:492] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: Bootstrap starting.
I20260619 12:05:06.942708 8261 tablet_bootstrap.cc:492] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704: No bootstrap required, opened a new log
I20260619 12:05:06.942770 8261 ts_tablet_manager.cc:1397] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:06.942962 8263 tablet_bootstrap.cc:654] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:06.942934 8261 raft_consensus.cc:348] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:06.942993 8261 raft_consensus.cc:374] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:06.943008 8261 raft_consensus.cc:729] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dfb4466a7ac441baa3bfd343cbf8704, State: Initialized, Role: FOLLOWER
I20260619 12:05:06.943054 8261 consensus_queue.cc:260] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:06.943257 8261 ts_tablet_manager.cc:1428] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:05:06.943601 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.943487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:58067)
0619 12:05:06.943528 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.943564 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 12:05:06.943958 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.943809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:55369)
0619 12:05:06.943876 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.943924 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:05:06.944113 7885 heartbeater.cc:503] Master 127.0.203.188:38511 was elected leader, sending a full tablet report...
I20260619 12:05:06.944136 8263 tablet_bootstrap.cc:492] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: No bootstrap required, opened a new log
I20260619 12:05:06.944242 8263 ts_tablet_manager.cc:1397] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:05:06.944438 8263 raft_consensus.cc:348] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:06.944533 8263 raft_consensus.cc:374] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:06.944553 8263 raft_consensus.cc:729] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23961998e5c7406f97ea2e6453f501f3, State: Initialized, Role: FOLLOWER
I20260619 12:05:06.944614 8263 consensus_queue.cc:260] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:06.944813 8263 ts_tablet_manager.cc:1428] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:06.945063 8262 tablet_bootstrap.cc:492] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: No bootstrap required, opened a new log
I20260619 12:05:06.945252 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.945132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:33703)
0619 12:05:06.945187 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.945223 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:05:06.945286 8262 ts_tablet_manager.cc:1397] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:05:06.945500 8262 raft_consensus.cc:348] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:06.945590 8262 raft_consensus.cc:374] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:06.945622 8262 raft_consensus.cc:729] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbd66f920e54163be6877706e7260a0, State: Initialized, Role: FOLLOWER
I20260619 12:05:06.945688 8262 consensus_queue.cc:260] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:06.945755 8158 heartbeater.cc:503] Master 127.0.203.188:38511 was elected leader, sending a full tablet report...
I20260619 12:05:06.945889 8262 ts_tablet_manager.cc:1428] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:05:06.945990 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.945651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:53035)
0619 12:05:06.945916 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.945950 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244}
I20260619 12:05:06.946408 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.946288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:52913)
0619 12:05:06.946337 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.946376 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 12:05:06.946753 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:06.946614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:43687)
0619 12:05:06.946699 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:06.946735 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 12:05:06.946794 8022 heartbeater.cc:503] Master 127.0.203.188:38511 was elected leader, sending a full tablet report...
I20260619 12:05:07.154410 8266 raft_consensus.cc:482] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:07.154520 8266 raft_consensus.cc:504] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:07.154875 8266 leader_election.cc:288] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efbd66f920e54163be6877706e7260a0 (127.0.203.130:34055), 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:07.159049 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.155096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:37085 (local address 127.0.203.129:53167)
0619 12:05:07.155159 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.155171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:07.155240 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:07.155476 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:07.155478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:07.155481 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:07.155549 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:07.155551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.156590 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.156591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:07.157905 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:07.157908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.158733 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.158734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:07.158819 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:07.158821 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:05:07.158883 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:05:07.158938 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":107}
I20260619 12:05:07.159077 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.155188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:53167 (local address 127.0.203.131:37085)
0619 12:05:07.155256 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:07.155258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:07.155273 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:05:07.155314 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:07.155315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:07.155327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:07.155448 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:07.155632 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.156464 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.158033 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.158635 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.158724 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:07.158739 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:07.158927 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 12:05:07.158985 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:05:07.159077 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.155273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:36333 (local address 127.0.203.130:34055)
0619 12:05:07.155321 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:07.155323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:07.155509 (+ 186us) server_negotiation.cc:407] Connection header received
0619 12:05:07.155545 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:07.155547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:07.155555 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:07.155609 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:07.155730 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.156499 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.158033 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.158634 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.158729 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:07.158739 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:07.158954 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 12:05:07.159007 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:05:07.159053 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.155152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34055 (local address 127.0.203.129:36333)
0619 12:05:07.155404 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.155421 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:07.155489 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:07.155628 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:07.155629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:07.155631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:07.155665 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:07.155666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.156602 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.156602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:07.157905 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:07.157908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.158734 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.158735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:07.158815 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:07.158821 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:05:07.158883 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:05:07.158938 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":6,"thread_start_us":150,"threads_started":1}
I20260619 12:05:07.159339 8112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fef0234f21c47da91192fbd35447eb2" candidate_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23961998e5c7406f97ea2e6453f501f3" is_pre_election: true
I20260619 12:05:07.159502 8112 raft_consensus.cc:2454] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dfb4466a7ac441baa3bfd343cbf8704 in term 0.
I20260619 12:05:07.159466 7976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fef0234f21c47da91192fbd35447eb2" candidate_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbd66f920e54163be6877706e7260a0" is_pre_election: true
I20260619 12:05:07.159647 7976 raft_consensus.cc:2454] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dfb4466a7ac441baa3bfd343cbf8704 in term 0.
I20260619 12:05:07.159716 7773 leader_election.cc:302] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dfb4466a7ac441baa3bfd343cbf8704, 23961998e5c7406f97ea2e6453f501f3; no voters:
I20260619 12:05:07.159857 8266 raft_consensus.cc:2781] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:07.159889 8266 raft_consensus.cc:482] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:07.159906 8266 raft_consensus.cc:3037] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:07.160629 8266 raft_consensus.cc:504] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:07.160789 8266 leader_election.cc:288] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [CANDIDATE]: Term 1 election: Requested vote from peers efbd66f920e54163be6877706e7260a0 (127.0.203.130:34055), 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085)
I20260619 12:05:07.161007 7976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fef0234f21c47da91192fbd35447eb2" candidate_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbd66f920e54163be6877706e7260a0"
I20260619 12:05:07.161079 7976 raft_consensus.cc:3037] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:07.161037 8112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fef0234f21c47da91192fbd35447eb2" candidate_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23961998e5c7406f97ea2e6453f501f3"
I20260619 12:05:07.161140 8112 raft_consensus.cc:3037] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:07.161896 7976 raft_consensus.cc:2454] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dfb4466a7ac441baa3bfd343cbf8704 in term 1.
I20260619 12:05:07.161952 8112 raft_consensus.cc:2454] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dfb4466a7ac441baa3bfd343cbf8704 in term 1.
I20260619 12:05:07.162125 7772 leader_election.cc:302] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dfb4466a7ac441baa3bfd343cbf8704, efbd66f920e54163be6877706e7260a0; no voters:
I20260619 12:05:07.162236 8266 raft_consensus.cc:2781] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:07.162459 8266 raft_consensus.cc:686] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 LEADER]: Becoming Leader. State: Replica: 0dfb4466a7ac441baa3bfd343cbf8704, State: Running, Role: LEADER
I20260619 12:05:07.162549 8266 consensus_queue.cc:237] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } }
I20260619 12:05:07.163333 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.163176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:51501)
0619 12:05:07.163258 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.163293 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 12:05:07.163570 7706 catalog_manager.cc:5495] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129). New cstate: current_term: 1 leader_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dfb4466a7ac441baa3bfd343cbf8704" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 42535 } health_report { overall_health: HEALTHY } } }
I20260619 12:05:07.171938 8259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.169173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.0.203.129:42535)
0619 12:05:07.169294 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:07.169296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:07.169308 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:05:07.169347 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:07.169348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:07.169372 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:07.169447 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:07.169629 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.170095 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.171058 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:07.171284 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:07.171395 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:07.171607 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:07.171733 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:07.171810 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:07.171838 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 12:05:07.171860 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260619 12:05:07.173285 8112 raft_consensus.cc:1261] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Refusing update from remote peer 0dfb4466a7ac441baa3bfd343cbf8704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:07.173395 7976 raft_consensus.cc:1261] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 1 FOLLOWER]: Refusing update from remote peer 0dfb4466a7ac441baa3bfd343cbf8704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:07.173705 8266 consensus_queue.cc:1035] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23961998e5c7406f97ea2e6453f501f3" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:07.173892 8271 consensus_queue.cc:1035] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efbd66f920e54163be6877706e7260a0" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:07.174805 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.174684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:44463)
0619 12:05:07.174734 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.174766 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 12:05:07.175402 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.175058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:44249)
0619 12:05:07.175293 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.175332 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260619 12:05:07.176205 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.176063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:42225)
0619 12:05:07.176132 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.176171 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 12:05:07.177312 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.177067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.131:36395)
0619 12:05:07.177242 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.177278 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260619 12:05:07.177604 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.177513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.129:53519)
0619 12:05:07.177549 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.177573 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 12:05:07.178422 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:07.178288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35585 (local address 127.0.203.130:32781)
0619 12:05:07.178362 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.178387 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:35585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 12:05:07.178954 7705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:07.179052 7705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:07.180554 7705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 622b770926624591a1afcea86866d2e9: Sending DeleteTablet for 3 replicas of tablet 6fef0234f21c47da91192fbd35447eb2
I20260619 12:05:07.180878 7956 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fef0234f21c47da91192fbd35447eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:07 UTC) from {username='oryx', principal='oryx/127.0.203.188@KRBTEST.COM'} at 127.0.0.1:34306
I20260619 12:05:07.180986 7819 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fef0234f21c47da91192fbd35447eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:07 UTC) from {username='oryx', principal='oryx/127.0.203.188@KRBTEST.COM'} at 127.0.0.1:36770
I20260619 12:05:07.181116 8284 tablet_replica.cc:330] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: stopping tablet replica
I20260619 12:05:07.181195 8284 raft_consensus.cc:2229] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:07.181252 8284 raft_consensus.cc:2258] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:07.181449 8285 tablet_replica.cc:330] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704: stopping tablet replica
I20260619 12:05:07.181429 8092 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fef0234f21c47da91192fbd35447eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:07 UTC) from {username='oryx', principal='oryx/127.0.203.188@KRBTEST.COM'} at 127.0.0.1:51878
I20260619 12:05:07.181509 8285 raft_consensus.cc:2229] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:05:07.181634 8284 ts_tablet_manager.cc:1905] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:07.181644 8285 raft_consensus.cc:2258] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:07.181994 8285 ts_tablet_manager.cc:1905] T 6fef0234f21c47da91192fbd35447eb2 P 0dfb4466a7ac441baa3bfd343cbf8704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:07.182391 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 7667
I20260619 12:05:07.186570 8284 ts_tablet_manager.cc:1918] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:07.186656 8284 log.cc:1196] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/6fef0234f21c47da91192fbd35447eb2
I20260619 12:05:07.186909 8284 ts_tablet_manager.cc:1939] T 6fef0234f21c47da91192fbd35447eb2 P efbd66f920e54163be6877706e7260a0: Deleting consensus metadata
I20260619 12:05:07.187423 7690 catalog_manager.cc:4841] TS efbd66f920e54163be6877706e7260a0 (127.0.203.130:34055): tablet 6fef0234f21c47da91192fbd35447eb2 (table test-table [id=83c10cfdbec94c1d9d9098de2fcd66b0]) successfully deleted
I20260619 12:05:07.189214 8286 tablet_replica.cc:330] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: stopping tablet replica
I20260619 12:05:07.189307 8286 raft_consensus.cc:2229] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:07.189376 8286 raft_consensus.cc:2258] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:05:07.189663 8258 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:07.189412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:42535 (local address 127.0.0.1:36788)
0619 12:05:07.189480 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:07.189494 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:07.189537 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:07.189601 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:42535: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:05:07.189816 7690 catalog_manager.cc:4584] TS 0dfb4466a7ac441baa3bfd343cbf8704 (127.0.203.129:42535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6fef0234f21c47da91192fbd35447eb2: Network error: Client connection negotiation failed: client connection to 127.0.203.129:42535: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260619 12:05:07.189888 8286 ts_tablet_manager.cc:1905] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:07.189949 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 7893
I20260619 12:05:07.191864 8286 ts_tablet_manager.cc:1918] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:07.191942 8286 log.cc:1196] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/6fef0234f21c47da91192fbd35447eb2
I20260619 12:05:07.192153 8286 ts_tablet_manager.cc:1939] T 6fef0234f21c47da91192fbd35447eb2 P 23961998e5c7406f97ea2e6453f501f3: Deleting consensus metadata
I20260619 12:05:07.192669 7691 catalog_manager.cc:4841] TS 23961998e5c7406f97ea2e6453f501f3 (127.0.203.131:37085): tablet 6fef0234f21c47da91192fbd35447eb2 (table test-table [id=83c10cfdbec94c1d9d9098de2fcd66b0]) successfully deleted
I20260619 12:05:07.196211 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 8029
I20260619 12:05:07.200693 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 7387
I20260619 12:05:07.205305 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 7574
2026-06-19T12:05:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7441 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:05:07 dist-test-slave-44vj krb5kdc[8289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:05:07 dist-test-slave-44vj krb5kdc[8289](info): set up 2 sockets
Jun 19 12:05:07 dist-test-slave-44vj krb5kdc[8289](info): commencing operation
krb5kdc: starting...
W20260619 12:05:08.192132 814 mini_kdc.cc:121] Time spent starting KDC: real 0.965s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:05:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab.
Entry for principal oryx/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab.
I20260619 12:05:08.318598 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42303
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab
--principal=oryx/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:33941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:42303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:08.407349 8302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:08.407660 8302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:08.407683 8302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:08.408958 8302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:08.409003 8302 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:05:08.409015 8302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:08.409027 8302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:08.409039 8302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:08.409049 8302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:08.410579 8302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:33941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:42303
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.190.keytab
--principal=oryx/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:08.410840 8302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:08.411064 8302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:05:08.417088 8302 server_base.cc:1034] running on GCE node
W20260619 12:05:08.418361 8310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:08.418313 8307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:08.418329 8308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:08.418808 8302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:08.419137 8302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:08.420315 8302 hybrid_clock.cc:648] HybridClock initialized: now 1781870708420279 us; error 48 us; skew 500 ppm
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:08.423179 8302 init.cc:377] Logged in from keytab as oryx/127.0.203.190@KRBTEST.COM (short username oryx)
I20260619 12:05:08.423707 8302 webserver.cc:460] Webserver started at http://127.0.203.190:34187/ using document root <none> and password file <none>
I20260619 12:05:08.423913 8302 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:08.423961 8302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:08.424060 8302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:08.425168 8302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "153d3e25c49d4e8da669c814e6640fb8"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "47c718b0854ca037b58fc5064fa645ef"
server_key_iv: "d571f62364a59c0b48ef90eef3730bf0"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.425489 8302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "153d3e25c49d4e8da669c814e6640fb8"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "47c718b0854ca037b58fc5064fa645ef"
server_key_iv: "d571f62364a59c0b48ef90eef3730bf0"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.426986 8302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:05:08.427852 8317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:08.428082 8302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:05:08.428184 8302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "153d3e25c49d4e8da669c814e6640fb8"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "47c718b0854ca037b58fc5064fa645ef"
server_key_iv: "d571f62364a59c0b48ef90eef3730bf0"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.428270 8302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:08.445942 8302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:08.448463 8302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:08.448613 8302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:08.454370 8302 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:42303
I20260619 12:05:08.454470 8379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:42303 every 8 connection(s)
I20260619 12:05:08.454854 8302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:05:08.455644 8380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:08.458406 8380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Bootstrap starting.
I20260619 12:05:08.459079 8380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:08.459439 8380 log.cc:824] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:08.460306 8380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: No bootstrap required, opened a new log
I20260619 12:05:08.462491 8380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153d3e25c49d4e8da669c814e6640fb8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42303 } }
I20260619 12:05:08.462671 8380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:08.462697 8380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153d3e25c49d4e8da669c814e6640fb8, State: Initialized, Role: FOLLOWER
I20260619 12:05:08.462865 8380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153d3e25c49d4e8da669c814e6640fb8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42303 } }
I20260619 12:05:08.462950 8380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:05:08.462975 8380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:05:08.463009 8380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:08.463742 8380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153d3e25c49d4e8da669c814e6640fb8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42303 } }
I20260619 12:05:08.463850 8380 leader_election.cc:302] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 153d3e25c49d4e8da669c814e6640fb8; no voters:
I20260619 12:05:08.464046 8380 leader_election.cc:288] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:05:08.464128 8383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:08.464349 8380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:08.464422 8383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 153d3e25c49d4e8da669c814e6640fb8, State: Running, Role: LEADER
I20260619 12:05:08.464588 8383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153d3e25c49d4e8da669c814e6640fb8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42303 } }
I20260619 12:05:08.464841 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 8302
I20260619 12:05:08.464967 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:05:08.465085 8385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 153d3e25c49d4e8da669c814e6640fb8. Latest consensus state: current_term: 1 leader_uuid: "153d3e25c49d4e8da669c814e6640fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153d3e25c49d4e8da669c814e6640fb8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42303 } } }
I20260619 12:05:08.465173 8385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:08.465189 814 external_mini_cluster.cc:1428] Setting key 6ded329aaf668a1d9fa5ef2c658c6fc5
I20260619 12:05:08.466105 8384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "153d3e25c49d4e8da669c814e6640fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153d3e25c49d4e8da669c814e6640fb8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42303 } } }
I20260619 12:05:08.466286 8384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:08.466167 8393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:08.467221 8393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:08.469671 8393 catalog_manager.cc:1347] Generated new cluster ID: d5a31ddf820f4176b1dfb119925ee818
I20260619 12:05:08.469739 8393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:08.478317 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.465734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.0.203.190:42303)
0619 12:05:08.466144 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:08.466157 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:08.466199 (+ 42us) server_negotiation.cc:407] Connection header received
0619 12:05:08.466365 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:08.466374 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:08.466449 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:08.466672 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:08.467548 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.468926 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.469822 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.470446 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.470550 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.473867 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:08.473875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:08.473876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:08.473896 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:08.476256 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.476773 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.476777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.476778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.476925 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.477133 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.477134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.477135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.477561 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:08.477756 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:08.477838 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 12:05:08.478020 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":275,"threads_started":1}
I20260619 12:05:08.491501 8393 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:05:08.492484 8393 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:08.497237 8393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Generated new TSK 0
I20260619 12:05:08.497462 8393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab.
Entry for principal oryx/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab.
I20260619 12:05:08.523969 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42303
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab
--principal=oryx/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:33941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:08.617229 8406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:08.617475 8406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:08.617511 8406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:08.618824 8406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:08.618886 8406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:08.618916 8406 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:05:08.618942 8406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:08.620700 8406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:33941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab
--principal=oryx/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:42303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:08.621021 8406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:08.621261 8406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:08.626875 8414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:08.626945 8412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:08.627013 8406 server_base.cc:1034] running on GCE node
W20260619 12:05:08.626875 8411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:08.627503 8406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:08.627810 8406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:08.629022 8406 hybrid_clock.cc:648] HybridClock initialized: now 1781870708628990 us; error 57 us; skew 500 ppm
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:08.632251 8406 init.cc:377] Logged in from keytab as oryx/127.0.203.129@KRBTEST.COM (short username oryx)
I20260619 12:05:08.632858 8406 webserver.cc:460] Webserver started at http://127.0.203.129:44161/ using document root <none> and password file <none>
I20260619 12:05:08.633111 8406 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:08.633167 8406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:08.633306 8406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:08.634449 8406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "31cb1552f79c4ac3847c6d0c3695784d"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "4765b6ad0b13a56c10b6bf88db1a3041"
server_key_iv: "35f56e2966e4cff56a75d38ff7d20d9e"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.634809 8406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "31cb1552f79c4ac3847c6d0c3695784d"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "4765b6ad0b13a56c10b6bf88db1a3041"
server_key_iv: "35f56e2966e4cff56a75d38ff7d20d9e"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.636447 8406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:05:08.637362 8421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:08.637583 8406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:08.637686 8406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "31cb1552f79c4ac3847c6d0c3695784d"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "4765b6ad0b13a56c10b6bf88db1a3041"
server_key_iv: "35f56e2966e4cff56a75d38ff7d20d9e"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.637758 8406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:08.649468 8406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:08.652264 8406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:08.652431 8406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:08.653188 8406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:08.653240 8406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:08.653280 8406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:08.653297 8406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:08.661067 8406 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:41039
I20260619 12:05:08.661242 8536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:41039 every 8 connection(s)
I20260619 12:05:08.661507 8406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:05:08.664162 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.653888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:51445 (local address 127.0.203.190:42303)
0619 12:05:08.653952 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:08.653955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:08.654242 (+ 287us) server_negotiation.cc:407] Connection header received
0619 12:05:08.655003 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:08.655007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:08.655034 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:08.655133 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:08.656040 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.656620 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.657719 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.657997 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.658060 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.660922 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:08.660924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:08.660925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:08.660935 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:08.662565 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.663041 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.663042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.663042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.663117 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.663348 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.663349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.663349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.663548 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:08.663610 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:08.663932 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 12:05:08.664057 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:05:08.664342 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.653849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42303 (local address 127.0.203.129:51445)
0619 12:05:08.654077 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:08.654129 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:08.654808 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:08.655241 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:08.655247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:08.655643 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:08.655922 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.655927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.656712 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.656713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.657637 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.657640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.658082 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.658083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.658242 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.659075 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:08.659096 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:08.660794 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:08.662673 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.662678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.662680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.662916 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.663189 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.663189 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.663189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.663294 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.663605 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:08.663609 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:08.663752 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 12:05:08.663858 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:05:08.664087 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
I20260619 12:05:08.665021 8537 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42303
I20260619 12:05:08.665177 8537 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:08.665441 8537 heartbeater.cc:511] Master 127.0.203.190:42303 requested a full tablet report, sending...
I20260619 12:05:08.666116 8333 ts_manager.cc:194] Registered new tserver with Master: 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039)
I20260619 12:05:08.666913 8333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:51445
I20260619 12:05:08.670861 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 8406
I20260619 12:05:08.671020 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:05:08.671278 814 external_mini_cluster.cc:1428] Setting key 6d4f9c8721398f463a9c95a2f1301a6b
I20260619 12:05:08.673003 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.666179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42303 (local address 127.0.203.129:39877)
0619 12:05:08.666323 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:08.666334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:08.666395 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:08.666565 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:08.666566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:08.666733 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:08.666814 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.666816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.667673 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.667674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.668330 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.668332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.668751 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.668752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.668825 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.669323 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:08.669329 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:08.669657 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:08.671769 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.671770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.671771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.671968 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.672246 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.672246 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.672247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.672319 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.672609 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:08.672611 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:08.672680 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 12:05:08.672759 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:05:08.672886 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260619 12:05:08.673053 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.666265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:39877 (local address 127.0.203.190:42303)
0619 12:05:08.666310 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:08.666312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:08.666393 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:05:08.666467 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:08.666468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:08.666481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:08.666561 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:08.666885 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.667562 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.668441 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.668640 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.668726 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.669773 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:08.669774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:08.669775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:08.669782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:08.671597 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.672089 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.672091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.672091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.672173 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.672376 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.672377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.672377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.672551 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:08.672610 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:08.672867 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 12:05:08.672972 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab.
Entry for principal oryx/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab.
I20260619 12:05:08.699864 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42303
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab
--principal=oryx/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:33941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:08.781466 8546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:08.781693 8546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:08.781719 8546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:08.782964 8546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:08.783025 8546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:08.783046 8546 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:05:08.783063 8546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:08.784709 8546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:33941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab
--principal=oryx/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:42303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:08.784976 8546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:08.785208 8546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:08.790642 8551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:08.790659 8554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:08.790843 8546 server_base.cc:1034] running on GCE node
W20260619 12:05:08.790884 8552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:08.791209 8546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:08.791559 8546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:08.792748 8546 hybrid_clock.cc:648] HybridClock initialized: now 1781870708792735 us; error 45 us; skew 500 ppm
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:08.795713 8546 init.cc:377] Logged in from keytab as oryx/127.0.203.130@KRBTEST.COM (short username oryx)
I20260619 12:05:08.796240 8546 webserver.cc:460] Webserver started at http://127.0.203.130:46735/ using document root <none> and password file <none>
I20260619 12:05:08.796484 8546 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:08.796532 8546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:08.796664 8546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:08.797907 8546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "e86b849dc6c74d3590cf97db8f63986b"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "c7e80b3b7a919d002bb974c9198364e2"
server_key_iv: "c46c995b2295b2b0260291c04cadaa94"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.798280 8546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "e86b849dc6c74d3590cf97db8f63986b"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "c7e80b3b7a919d002bb974c9198364e2"
server_key_iv: "c46c995b2295b2b0260291c04cadaa94"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.799901 8546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:05:08.800992 8561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:08.801311 8546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:08.801425 8546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "e86b849dc6c74d3590cf97db8f63986b"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "c7e80b3b7a919d002bb974c9198364e2"
server_key_iv: "c46c995b2295b2b0260291c04cadaa94"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.801534 8546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:08.813015 8546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:08.815992 8546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:08.816159 8546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:08.816825 8546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:08.816876 8546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:08.816910 8546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:08.816921 8546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:08.825421 8546 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:34725
I20260619 12:05:08.825788 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:34725 every 8 connection(s)
I20260619 12:05:08.825940 8546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:08.828509 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.817463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:40689 (local address 127.0.203.190:42303)
0619 12:05:08.817513 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:08.817515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:08.817844 (+ 329us) server_negotiation.cc:407] Connection header received
0619 12:05:08.818396 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:08.818398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:08.818416 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:08.818501 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:08.819513 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.820019 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.821087 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.821297 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.821356 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.825209 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:08.825212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:08.825212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:08.825221 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:08.826860 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.827335 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.827336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.827337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.827476 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.827780 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.827781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.827781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.827910 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:08.827960 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:08.828335 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 12:05:08.828436 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:05:08.828852 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.817491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42303 (local address 127.0.203.130:40689)
0619 12:05:08.817669 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:08.817712 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:08.818266 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:08.818589 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:08.818593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:08.819034 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:08.819394 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.819401 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.820110 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.820112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.820993 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.820996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.821373 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.821374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.821513 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.822122 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:08.822138 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:08.825074 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:08.826984 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.826991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.826993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.827254 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.827570 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.827571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.827571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.827705 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.828021 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:08.828027 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:08.828199 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 12:05:08.828313 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:05:08.828601 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260619 12:05:08.829520 8677 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42303
I20260619 12:05:08.829623 8677 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:08.829851 8677 heartbeater.cc:511] Master 127.0.203.190:42303 requested a full tablet report, sending...
I20260619 12:05:08.830295 8333 ts_manager.cc:194] Registered new tserver with Master: e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725)
I20260619 12:05:08.830838 8333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:40689
I20260619 12:05:08.835886 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 8546
I20260619 12:05:08.836027 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:05:08.836237 814 external_mini_cluster.cc:1428] Setting key edc2211150bbb72a01935ee333a94ec8
I20260619 12:05:08.836823 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.830689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42303 (local address 127.0.203.130:50759)
0619 12:05:08.830866 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:08.830877 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:08.830939 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:08.831132 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:08.831133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:08.831320 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:08.831443 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.831445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.832199 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.832200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.832771 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.832774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.833092 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.833093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.833158 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.833614 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:08.833620 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:08.834040 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:08.835601 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.835602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.835603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.835804 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.836010 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:08.836010 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:08.836011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:08.836074 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:08.836409 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:08.836410 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:08.836490 (+ 80us) client_negotiation.cc:769] Sending connection context
0619 12:05:08.836588 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:05:08.836728 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":123,"threads_started":1}
I20260619 12:05:08.836814 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.830756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:50759 (local address 127.0.203.190:42303)
0619 12:05:08.830993 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:08.830995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:08.831005 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:08.831024 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:08.831025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:08.831036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:08.831115 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:08.831530 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.832083 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.832849 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.833031 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.833089 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:08.834154 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:08.834156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:08.834157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:08.834165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:08.835499 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.835880 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.835882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.835882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.835939 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:08.836148 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:08.836150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:08.836150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:08.836318 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:08.836393 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:08.836623 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 12:05:08.836709 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
WARNING: no policy specified for oryx/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab.
Entry for principal oryx/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab.
I20260619 12:05:08.866241 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42303
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab
--principal=oryx/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:33941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:08.957043 8686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:08.957306 8686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:08.957345 8686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:08.958695 8686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:08.958746 8686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:08.958777 8686 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:05:08.958801 8686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:08.960547 8686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:33941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab
--principal=oryx/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.203.190:42303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:08.960829 8686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:08.961074 8686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:08.966639 8691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:08.966589 8692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:08.966943 8686 server_base.cc:1034] running on GCE node
W20260619 12:05:08.966677 8694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:08.967353 8686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:08.967698 8686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:08.968878 8686 hybrid_clock.cc:648] HybridClock initialized: now 1781870708968832 us; error 64 us; skew 500 ppm
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:08.972084 8686 init.cc:377] Logged in from keytab as oryx/127.0.203.131@KRBTEST.COM (short username oryx)
I20260619 12:05:08.972719 8686 webserver.cc:460] Webserver started at http://127.0.203.131:37835/ using document root <none> and password file <none>
I20260619 12:05:08.972990 8686 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:08.973047 8686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:08.973193 8686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:08.974480 8686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "e115141793984654a21f10d69f29fdc8"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "dfdcafba90e28579e213edb976cb82aa"
server_key_iv: "92bf8b5988396e7a483284a854198a32"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.974880 8686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "e115141793984654a21f10d69f29fdc8"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "dfdcafba90e28579e213edb976cb82aa"
server_key_iv: "92bf8b5988396e7a483284a854198a32"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.976523 8686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:05:08.977594 8701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:08.977896 8686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:08.977993 8686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "e115141793984654a21f10d69f29fdc8"
format_stamp: "Formatted at 2026-06-19 12:05:08 on dist-test-slave-44vj"
server_key: "dfdcafba90e28579e213edb976cb82aa"
server_key_iv: "92bf8b5988396e7a483284a854198a32"
server_key_version: "encryptionkey@0"
I20260619 12:05:08.978083 8686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:08.991431 8686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:08.993971 8686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:08.994139 8686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:08.994858 8686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:08.994912 8686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:08.994944 8686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:08.994961 8686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:05:08 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:09.003504 8686 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:40001
I20260619 12:05:09.003782 8816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:40001 every 8 connection(s)
I20260619 12:05:09.004016 8686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:05:09.006099 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.995462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:39339 (local address 127.0.203.190:42303)
0619 12:05:08.995546 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:08.995549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:08.995889 (+ 340us) server_negotiation.cc:407] Connection header received
0619 12:05:08.996779 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:08.996781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:08.996797 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:08.996937 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:08.997849 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.998378 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.999537 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.999748 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.999805 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.002709 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:09.002711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:09.002711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:09.002720 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:09.004374 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.004898 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.004901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.004902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.004991 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.005341 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.005341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.005342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.005502 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:09.005567 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.005898 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.006004 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:05:09.006352 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:08.995450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42303 (local address 127.0.203.131:39339)
0619 12:05:08.995691 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:08.995754 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:08.996639 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:08.997028 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:08.997034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:08.997462 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:08.997715 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.997723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.998481 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.998482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.999442 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:08.999445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:08.999826 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:08.999827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:08.999972 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.000736 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:09.000747 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:09.002587 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:09.004490 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:09.004496 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:09.004498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:09.004795 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:09.005108 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:09.005110 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:09.005110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:09.005252 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:09.005592 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:09.005598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:09.005752 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.005864 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.006021 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":117,"threads_started":1}
I20260619 12:05:09.007030 8817 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42303
I20260619 12:05:09.007151 8817 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:09.007478 8817 heartbeater.cc:511] Master 127.0.203.190:42303 requested a full tablet report, sending...
I20260619 12:05:09.008064 8333 ts_manager.cc:194] Registered new tserver with Master: e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.008585 8333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:39339
I20260619 12:05:09.012583 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 8686
I20260619 12:05:09.012704 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:05:09.012935 814 external_mini_cluster.cc:1428] Setting key f5f68590bac8af53c839c7935ce1a880
I20260619 12:05:09.014235 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:05:09.015038 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.008387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:38675 (local address 127.0.203.190:42303)
0619 12:05:09.008439 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.008441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.008600 (+ 159us) server_negotiation.cc:407] Connection header received
0619 12:05:09.008658 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.008659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.008671 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.008718 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:09.009193 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.009708 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.010429 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.010656 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.010715 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.012093 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:09.012096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:09.012096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:09.012104 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:09.013713 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.014093 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.014095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.014095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.014222 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.014477 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.014478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.014478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.014620 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:09.014684 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.014867 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.014962 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:05:09.015187 8823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.008327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42303 (local address 127.0.203.131:38675)
0619 12:05:09.008520 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.008532 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.008599 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.008771 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.008772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.009029 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:09.009115 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.009118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.009800 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.009801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.010353 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.010355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.010720 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.010721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.010786 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.011551 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:09.011558 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:09.011928 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:09.013781 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:09.013783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:09.013783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:09.013980 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:09.014320 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:09.014321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:09.014321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:09.014406 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:09.014683 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:09.014684 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:09.014755 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.014842 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.015059 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":98,"threads_started":1}
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:05:09.039863 8395 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:09.033438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.0.203.190:42303)
0619 12:05:09.033527 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.033530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.033611 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:05:09.033679 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.033680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.033695 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.033754 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:09.034321 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.034861 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.035720 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.035923 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.035975 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.039756 (+ 3781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60264: BlockingRecv error: recv got EOF from 127.0.0.1:60264 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:09.049494 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.041127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.0.203.190:42303)
0619 12:05:09.041234 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.041236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.041307 (+ 71us) server_negotiation.cc:407] Connection header received
0619 12:05:09.041386 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.041388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.041400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.041449 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:09.042010 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.042517 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.043197 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.043426 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.043488 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.046256 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:09.046258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:09.046258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:09.046289 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:09.047859 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.048425 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.048427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.048427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.048512 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.048802 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.048805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.048805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.048980 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:09.049089 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.049326 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.049417 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 12:05:09.051644 8333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:05:09.053361 8333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:09.061964 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.056906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34725 (local address 127.0.0.1:59204)
0619 12:05:09.057617 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.057628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.057703 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.057925 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.057927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.057930 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.058005 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.058009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.059003 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.059005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.060313 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.060317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.061565 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.061566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.061663 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.061693 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.061787 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.061849 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":72,"threads_started":1}
I20260619 12:05:09.061993 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.057136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.0.203.130:34725)
0619 12:05:09.057305 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.057310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.057704 (+ 394us) server_negotiation.cc:407] Connection header received
0619 12:05:09.057801 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.057804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.057821 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.057913 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.058116 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.058899 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.060484 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.061439 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.061555 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.061665 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.061804 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.061850 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260619 12:05:09.062598 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.056649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:40001 (local address 127.0.0.1:58196)
0619 12:05:09.057097 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.057260 (+ 163us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.057463 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.058016 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.058017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.058019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.058048 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.058050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.060464 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.060464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.061253 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.061256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.062360 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.062361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.062429 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.062433 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.062477 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.062521 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":80,"threads_started":1}
I20260619 12:05:09.062882 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.056888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.0.203.131:40001)
0619 12:05:09.057343 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.057347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.057362 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:05:09.057832 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.057837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.057855 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.057915 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.058139 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.059267 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.061354 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.062152 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.062241 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.062325 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.062730 (+ 405us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.062782 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":92,"threads_started":1}
I20260619 12:05:09.063205 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.056837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.0.203.129:41039)
0619 12:05:09.057141 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.057217 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.057513 (+ 296us) server_negotiation.cc:407] Connection header received
0619 12:05:09.057611 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.057614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.057635 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.057727 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.058026 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.059267 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.060719 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.061476 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.061555 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.061664 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.063054 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.063119 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":187,"threads_started":1}
I20260619 12:05:09.062937 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.056656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41039 (local address 127.0.0.1:38916)
0619 12:05:09.057418 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.057430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.057516 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.057826 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.057830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.057834 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.057959 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.057962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.059567 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.059568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.060608 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.060610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.062610 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.062611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.062773 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.062778 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.062835 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.062876 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":98,"threads_started":1}
I20260619 12:05:09.063488 8611 tablet_service.cc:1467] Processing CreateTablet for tablet 51f22bff4fe74cec89fdd59a65b498f0 (DEFAULT_TABLE table=test-table [id=e625cf109c9141a9b05392f1f7a7f91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.063977 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51f22bff4fe74cec89fdd59a65b498f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.064188 8471 tablet_service.cc:1467] Processing CreateTablet for tablet 51f22bff4fe74cec89fdd59a65b498f0 (DEFAULT_TABLE table=test-table [id=e625cf109c9141a9b05392f1f7a7f91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.064639 8471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51f22bff4fe74cec89fdd59a65b498f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.067971 8845 tablet_bootstrap.cc:492] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Bootstrap starting.
I20260619 12:05:09.068470 8846 tablet_bootstrap.cc:492] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Bootstrap starting.
I20260619 12:05:09.068768 8845 tablet_bootstrap.cc:654] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.069157 8846 tablet_bootstrap.cc:654] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.069144 8845 log.cc:824] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:09.069134 8751 tablet_service.cc:1467] Processing CreateTablet for tablet 51f22bff4fe74cec89fdd59a65b498f0 (DEFAULT_TABLE table=test-table [id=e625cf109c9141a9b05392f1f7a7f91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.069489 8846 log.cc:824] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:09.069545 8751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51f22bff4fe74cec89fdd59a65b498f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.070236 8845 tablet_bootstrap.cc:492] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: No bootstrap required, opened a new log
I20260619 12:05:09.070324 8845 ts_tablet_manager.cc:1397] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:05:09.072000 8846 tablet_bootstrap.cc:492] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: No bootstrap required, opened a new log
I20260619 12:05:09.072077 8846 ts_tablet_manager.cc:1397] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:05:09.072525 8845 raft_consensus.cc:348] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.072736 8845 raft_consensus.cc:374] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.072770 8845 raft_consensus.cc:729] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86b849dc6c74d3590cf97db8f63986b, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.072924 8845 consensus_queue.cc:260] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.073264 8845 ts_tablet_manager.cc:1428] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:05:09.073343 8677 heartbeater.cc:503] Master 127.0.203.190:42303 was elected leader, sending a full tablet report...
I20260619 12:05:09.073820 8846 raft_consensus.cc:348] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.073964 8846 raft_consensus.cc:374] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.073994 8846 raft_consensus.cc:729] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cb1552f79c4ac3847c6d0c3695784d, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.074119 8846 consensus_queue.cc:260] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.074438 8846 ts_tablet_manager.cc:1428] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:05:09.074426 8850 tablet_bootstrap.cc:492] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Bootstrap starting.
I20260619 12:05:09.074612 8537 heartbeater.cc:503] Master 127.0.203.190:42303 was elected leader, sending a full tablet report...
I20260619 12:05:09.075186 8850 tablet_bootstrap.cc:654] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.075515 8850 log.cc:824] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:09.076448 8850 tablet_bootstrap.cc:492] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: No bootstrap required, opened a new log
I20260619 12:05:09.076529 8850 ts_tablet_manager.cc:1397] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
W20260619 12:05:09.077224 8678 tablet.cc:2369] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:05:09.078058 8850 raft_consensus.cc:348] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.078212 8850 raft_consensus.cc:374] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.078244 8850 raft_consensus.cc:729] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e115141793984654a21f10d69f29fdc8, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.078361 8850 consensus_queue.cc:260] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.078634 8850 ts_tablet_manager.cc:1428] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 12:05:09.078702 8817 heartbeater.cc:503] Master 127.0.203.190:42303 was elected leader, sending a full tablet report...
W20260619 12:05:09.162569 8538 tablet.cc:2369] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:05:09.254805 8818 tablet.cc:2369] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:05:09.318387 8851 raft_consensus.cc:482] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:09.318531 8851 raft_consensus.cc:504] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.318925 8851 leader_election.cc:288] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.323653 8855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.319232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:40001 (local address 127.0.203.129:39875)
0619 12:05:09.319463 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.319478 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.319577 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.319774 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.319775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.319778 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.319889 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.319892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.320974 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.320976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.322397 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.322415 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.323293 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.323293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.323398 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.323400 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.323451 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.323514 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":104,"thread_start_us":98,"threads_started":1}
I20260619 12:05:09.323684 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.319407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:54477 (local address 127.0.203.130:34725)
0619 12:05:09.319480 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.319483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.319622 (+ 139us) server_negotiation.cc:407] Connection header received
0619 12:05:09.319654 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.319656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.319669 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.319757 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.320052 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.320823 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.322550 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.323159 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.323254 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.323269 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.323511 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.323596 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:05:09.323683 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.319407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:39875 (local address 127.0.203.131:40001)
0619 12:05:09.319462 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.319464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.319600 (+ 136us) server_negotiation.cc:407] Connection header received
0619 12:05:09.319653 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.319655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.319669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.319757 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.320026 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.320818 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.322550 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.323218 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.323286 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.323295 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.323511 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.323596 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:05:09.323658 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.319234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34725 (local address 127.0.203.129:54477)
0619 12:05:09.319517 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.319526 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.319590 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.319773 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.319774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.319776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.319944 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.319946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.321011 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.321012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.322397 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.322401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.323267 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.323269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.323352 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.323387 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.323451 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.323514 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":40,"spinlock_wait_cycles":35584,"thread_start_us":146,"threads_started":1}
I20260619 12:05:09.324138 8771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f22bff4fe74cec89fdd59a65b498f0" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8" is_pre_election: true
I20260619 12:05:09.324226 8631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f22bff4fe74cec89fdd59a65b498f0" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86b849dc6c74d3590cf97db8f63986b" is_pre_election: true
I20260619 12:05:09.324352 8771 raft_consensus.cc:2454] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 0.
I20260619 12:05:09.324397 8631 raft_consensus.cc:2454] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 0.
I20260619 12:05:09.324595 8422 leader_election.cc:302] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31cb1552f79c4ac3847c6d0c3695784d, e115141793984654a21f10d69f29fdc8; no voters:
I20260619 12:05:09.324796 8851 raft_consensus.cc:2781] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:09.324872 8851 raft_consensus.cc:482] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:09.324892 8851 raft_consensus.cc:3037] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.325747 8851 raft_consensus.cc:504] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.325899 8851 leader_election.cc:288] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 election: Requested vote from peers e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.326192 8771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f22bff4fe74cec89fdd59a65b498f0" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8"
I20260619 12:05:09.326320 8771 raft_consensus.cc:3037] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.327317 8771 raft_consensus.cc:2454] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 1.
I20260619 12:05:09.327520 8631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f22bff4fe74cec89fdd59a65b498f0" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86b849dc6c74d3590cf97db8f63986b"
I20260619 12:05:09.327634 8422 leader_election.cc:302] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31cb1552f79c4ac3847c6d0c3695784d, e115141793984654a21f10d69f29fdc8; no voters:
I20260619 12:05:09.327658 8631 raft_consensus.cc:3037] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.327754 8851 raft_consensus.cc:2781] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:09.327914 8851 raft_consensus.cc:686] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 LEADER]: Becoming Leader. State: Replica: 31cb1552f79c4ac3847c6d0c3695784d, State: Running, Role: LEADER
I20260619 12:05:09.328048 8851 consensus_queue.cc:237] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.328891 8631 raft_consensus.cc:2454] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 1.
I20260619 12:05:09.329129 8333 catalog_manager.cc:5495] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d reported cstate change: term changed from 0 to 1, leader changed from <none> to 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129). New cstate: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } health_report { overall_health: UNKNOWN } } }
I20260619 12:05:09.353070 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.350477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38926 (local address 127.0.203.129:41039)
0619 12:05:09.350577 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.350579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.350589 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:09.350663 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.350664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.350686 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.350751 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.350918 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.351484 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.352257 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.352466 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.352531 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.352714 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.352827 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.352905 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.352945 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.352968 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 12:05:09.355322 8771 raft_consensus.cc:1261] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Refusing update from remote peer 31cb1552f79c4ac3847c6d0c3695784d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.355341 8631 raft_consensus.cc:1261] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Refusing update from remote peer 31cb1552f79c4ac3847c6d0c3695784d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.355736 8851 consensus_queue.cc:1035] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.355930 8856 consensus_queue.cc:1035] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.359220 8860 mvcc.cc:205] Tried to move back new op lower bound from 7298542425516064768 to 7298542425408536576. Current Snapshot: MvccSnapshot[applied={T|T < 7298542425516064768}]
I20260619 12:05:09.359623 8861 mvcc.cc:205] Tried to move back new op lower bound from 7298542425516064768 to 7298542425408536576. Current Snapshot: MvccSnapshot[applied={T|T < 7298542425516064768}]
I20260619 12:05:09.361439 8333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.361567 8333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.362455 8333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Sending DeleteTablet for 3 replicas of tablet 51f22bff4fe74cec89fdd59a65b498f0
I20260619 12:05:09.362882 8471 tablet_service.cc:1514] Processing DeleteTablet for tablet 51f22bff4fe74cec89fdd59a65b498f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:38916
I20260619 12:05:09.362938 8751 tablet_service.cc:1514] Processing DeleteTablet for tablet 51f22bff4fe74cec89fdd59a65b498f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:58196
I20260619 12:05:09.363117 8333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:05:09.363296 8333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:09.363466 8869 tablet_replica.cc:330] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: stopping tablet replica
I20260619 12:05:09.363610 8869 raft_consensus.cc:2229] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:05:09.363732 8869 raft_consensus.cc:2258] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.364220 8869 ts_tablet_manager.cc:1905] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.365840 8869 ts_tablet_manager.cc:1918] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:09.365904 8869 log.cc:1196] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/51f22bff4fe74cec89fdd59a65b498f0
I20260619 12:05:09.366108 8869 ts_tablet_manager.cc:1939] T 51f22bff4fe74cec89fdd59a65b498f0 P 31cb1552f79c4ac3847c6d0c3695784d: Deleting consensus metadata
I20260619 12:05:09.366524 8319 catalog_manager.cc:4841] TS 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039): tablet 51f22bff4fe74cec89fdd59a65b498f0 (table test-table [id=e625cf109c9141a9b05392f1f7a7f91f]) successfully deleted
I20260619 12:05:09.367604 8611 tablet_service.cc:1514] Processing DeleteTablet for tablet 51f22bff4fe74cec89fdd59a65b498f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:59204
I20260619 12:05:09.367835 8870 tablet_replica.cc:330] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: stopping tablet replica
I20260619 12:05:09.367987 8870 raft_consensus.cc:2229] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:09.368150 8870 raft_consensus.cc:2258] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.368897 8870 ts_tablet_manager.cc:1905] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.370044 8471 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5fe501375e4cdb956f693029dbeeeb (DEFAULT_TABLE table=test-table [id=04e2b7664852477fa932e679d09676ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.370165 8471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5fe501375e4cdb956f693029dbeeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.371248 8870 ts_tablet_manager.cc:1918] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:09.371336 8870 log.cc:1196] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/51f22bff4fe74cec89fdd59a65b498f0
I20260619 12:05:09.371450 8611 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5fe501375e4cdb956f693029dbeeeb (DEFAULT_TABLE table=test-table [id=04e2b7664852477fa932e679d09676ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.371541 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5fe501375e4cdb956f693029dbeeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.371752 8870 ts_tablet_manager.cc:1939] T 51f22bff4fe74cec89fdd59a65b498f0 P e86b849dc6c74d3590cf97db8f63986b: Deleting consensus metadata
I20260619 12:05:09.371824 8871 tablet_replica.cc:330] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: stopping tablet replica
I20260619 12:05:09.372015 8871 raft_consensus.cc:2229] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:09.372174 8871 raft_consensus.cc:2258] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.372855 8845 tablet_bootstrap.cc:492] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: Bootstrap starting.
I20260619 12:05:09.373034 8319 catalog_manager.cc:4841] TS e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725): tablet 51f22bff4fe74cec89fdd59a65b498f0 (table test-table [id=e625cf109c9141a9b05392f1f7a7f91f]) successfully deleted
I20260619 12:05:09.373081 8871 ts_tablet_manager.cc:1905] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.373245 8751 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5fe501375e4cdb956f693029dbeeeb (DEFAULT_TABLE table=test-table [id=04e2b7664852477fa932e679d09676ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.373384 8751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5fe501375e4cdb956f693029dbeeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.373387 8845 tablet_bootstrap.cc:654] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.374094 8845 tablet_bootstrap.cc:492] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: No bootstrap required, opened a new log
I20260619 12:05:09.374152 8845 ts_tablet_manager.cc:1397] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:09.374325 8845 raft_consensus.cc:348] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.374396 8845 raft_consensus.cc:374] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.374413 8845 raft_consensus.cc:729] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86b849dc6c74d3590cf97db8f63986b, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.374470 8845 consensus_queue.cc:260] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.374752 8845 ts_tablet_manager.cc:1428] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:05:09.375293 8850 tablet_bootstrap.cc:492] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: Bootstrap starting.
I20260619 12:05:09.375448 8871 ts_tablet_manager.cc:1918] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:09.375540 8871 log.cc:1196] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/51f22bff4fe74cec89fdd59a65b498f0
I20260619 12:05:09.375756 8850 tablet_bootstrap.cc:654] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.375844 8871 ts_tablet_manager.cc:1939] T 51f22bff4fe74cec89fdd59a65b498f0 P e115141793984654a21f10d69f29fdc8: Deleting consensus metadata
I20260619 12:05:09.376436 8318 catalog_manager.cc:4841] TS e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001): tablet 51f22bff4fe74cec89fdd59a65b498f0 (table test-table [id=e625cf109c9141a9b05392f1f7a7f91f]) successfully deleted
I20260619 12:05:09.376659 8850 tablet_bootstrap.cc:492] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: No bootstrap required, opened a new log
I20260619 12:05:09.376713 8850 ts_tablet_manager.cc:1397] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:09.376853 8850 raft_consensus.cc:348] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.376919 8850 raft_consensus.cc:374] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.376930 8850 raft_consensus.cc:729] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e115141793984654a21f10d69f29fdc8, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.376973 8850 consensus_queue.cc:260] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.377061 8850 ts_tablet_manager.cc:1428] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:09.377571 8846 tablet_bootstrap.cc:492] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: Bootstrap starting.
I20260619 12:05:09.378062 8846 tablet_bootstrap.cc:654] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.378854 8846 tablet_bootstrap.cc:492] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: No bootstrap required, opened a new log
I20260619 12:05:09.378904 8846 ts_tablet_manager.cc:1397] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:09.379073 8846 raft_consensus.cc:348] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.379165 8846 raft_consensus.cc:374] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.379184 8846 raft_consensus.cc:729] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cb1552f79c4ac3847c6d0c3695784d, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.379240 8846 consensus_queue.cc:260] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.379338 8846 ts_tablet_manager.cc:1428] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:09.488164 8849 raft_consensus.cc:482] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:09.488277 8849 raft_consensus.cc:504] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.488643 8849 leader_election.cc:288] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.491923 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.488988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:49119 (local address 127.0.203.131:40001)
0619 12:05:09.489033 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.489036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.489308 (+ 272us) server_negotiation.cc:407] Connection header received
0619 12:05:09.489401 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.489402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.489411 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.489457 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.489626 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.490111 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.491132 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.491547 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.491618 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.491630 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.491790 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.491840 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:05:09.491931 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.489057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:59935 (local address 127.0.203.129:41039)
0619 12:05:09.489105 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.489107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.489118 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:09.489150 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.489151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.489166 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.489251 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.489447 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.489961 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.491114 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.491544 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.491614 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.491627 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.491777 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.491827 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:05:09.492183 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.488853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41039 (local address 127.0.203.130:59935)
0619 12:05:09.488993 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.489008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.489087 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.489267 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.489268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.489270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.489359 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.489361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.490090 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.490091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.491013 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.491016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.491650 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.491651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.491715 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.491719 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.491767 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.491842 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260619 12:05:09.492224 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.488925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:40001 (local address 127.0.203.130:49119)
0619 12:05:09.489243 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.489257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.489335 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.489469 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.489470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.489472 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.489565 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.489567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.490188 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.490189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.491035 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.491038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.491642 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.491643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.491723 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.491728 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.491776 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.491870 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":28,"thread_start_us":225,"threads_started":1}
I20260619 12:05:09.492493 8491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5fe501375e4cdb956f693029dbeeeb" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cb1552f79c4ac3847c6d0c3695784d" is_pre_election: true
I20260619 12:05:09.492676 8491 raft_consensus.cc:2454] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 0.
I20260619 12:05:09.492687 8771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5fe501375e4cdb956f693029dbeeeb" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8" is_pre_election: true
I20260619 12:05:09.492770 8771 raft_consensus.cc:2454] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 0.
I20260619 12:05:09.492871 8563 leader_election.cc:302] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31cb1552f79c4ac3847c6d0c3695784d, e86b849dc6c74d3590cf97db8f63986b; no voters:
I20260619 12:05:09.493049 8849 raft_consensus.cc:2781] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:09.493098 8849 raft_consensus.cc:482] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:09.493119 8849 raft_consensus.cc:3037] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.493820 8849 raft_consensus.cc:504] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.493965 8849 leader_election.cc:288] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 election: Requested vote from peers 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.494113 8771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5fe501375e4cdb956f693029dbeeeb" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8"
I20260619 12:05:09.494194 8771 raft_consensus.cc:3037] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.494170 8491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5fe501375e4cdb956f693029dbeeeb" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cb1552f79c4ac3847c6d0c3695784d"
I20260619 12:05:09.494262 8491 raft_consensus.cc:3037] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.494894 8771 raft_consensus.cc:2454] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 1.
I20260619 12:05:09.495110 8491 raft_consensus.cc:2454] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 1.
I20260619 12:05:09.495128 8562 leader_election.cc:302] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e115141793984654a21f10d69f29fdc8, e86b849dc6c74d3590cf97db8f63986b; no voters:
I20260619 12:05:09.495334 8849 raft_consensus.cc:2781] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:09.495646 8849 raft_consensus.cc:686] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 1 LEADER]: Becoming Leader. State: Replica: e86b849dc6c74d3590cf97db8f63986b, State: Running, Role: LEADER
I20260619 12:05:09.495730 8849 consensus_queue.cc:237] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.496645 8332 catalog_manager.cc:5495] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b reported cstate change: term changed from 0 to 1, leader changed from <none> to e86b849dc6c74d3590cf97db8f63986b (127.0.203.130). New cstate: current_term: 1 leader_uuid: "e86b849dc6c74d3590cf97db8f63986b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } health_report { overall_health: UNKNOWN } } }
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870708, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.190@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:09.575626 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.568384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.0.203.190:42303)
0619 12:05:09.568497 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.568499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.568559 (+ 60us) server_negotiation.cc:407] Connection header received
0619 12:05:09.568651 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.568652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.568665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.568727 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:09.569281 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.569816 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.570475 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.570669 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.570726 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.573141 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:09.573143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:09.573143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:09.573151 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:09.574440 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.574801 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.574802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.574802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.574865 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.575025 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.575026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.575026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.575150 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:09.575210 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.575347 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.575479 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260619 12:05:09.575678 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.568289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42303 (local address 127.0.0.1:60286)
0619 12:05:09.568430 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.568444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.568589 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.568735 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.568736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.569131 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:09.569195 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.569198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.569885 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.569886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.570406 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.570407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.570728 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.570728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.570774 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.571404 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:09.571411 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:09.573048 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:09.574526 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:09.574534 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:09.574538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:09.574736 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:09.574918 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:09.574919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:09.574919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:09.574976 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:09.575203 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:09.575211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:09.575261 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.575320 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.575548 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":75,"thread_start_us":77,"threads_started":1}
I20260619 12:05:09.578126 8332 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:60286:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:05:09.578353 8332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:09.580090 8611 tablet_service.cc:1467] Processing CreateTablet for tablet a5baf37a5893455e8f267637271fbb11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed8d8374f4d14039ad982dcd42f46329]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:05:09.580127 8471 tablet_service.cc:1467] Processing CreateTablet for tablet a5baf37a5893455e8f267637271fbb11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed8d8374f4d14039ad982dcd42f46329]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:05:09.580281 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5baf37a5893455e8f267637271fbb11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.580291 8471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5baf37a5893455e8f267637271fbb11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.581532 8845 tablet_bootstrap.cc:492] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: Bootstrap starting.
I20260619 12:05:09.581604 8751 tablet_service.cc:1467] Processing CreateTablet for tablet a5baf37a5893455e8f267637271fbb11 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed8d8374f4d14039ad982dcd42f46329]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:05:09.581755 8751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5baf37a5893455e8f267637271fbb11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.581758 8846 tablet_bootstrap.cc:492] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: Bootstrap starting.
I20260619 12:05:09.582005 8845 tablet_bootstrap.cc:654] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.582160 8846 tablet_bootstrap.cc:654] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.582731 8845 tablet_bootstrap.cc:492] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: No bootstrap required, opened a new log
I20260619 12:05:09.582785 8845 ts_tablet_manager.cc:1397] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:09.582954 8845 raft_consensus.cc:348] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.583027 8845 raft_consensus.cc:374] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.583046 8845 raft_consensus.cc:729] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86b849dc6c74d3590cf97db8f63986b, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.583092 8845 consensus_queue.cc:260] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.583196 8845 ts_tablet_manager.cc:1428] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:09.583179 8850 tablet_bootstrap.cc:492] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: Bootstrap starting.
I20260619 12:05:09.583307 8849 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.583580 8849 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:05:09.583662 8850 tablet_bootstrap.cc:654] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.583842 8846 tablet_bootstrap.cc:492] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: No bootstrap required, opened a new log
I20260619 12:05:09.583891 8846 ts_tablet_manager.cc:1397] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:05:09.584002 8846 raft_consensus.cc:348] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.584085 8846 raft_consensus.cc:374] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.584116 8846 raft_consensus.cc:729] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cb1552f79c4ac3847c6d0c3695784d, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.584174 8846 consensus_queue.cc:260] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.584275 8846 ts_tablet_manager.cc:1428] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:05:09.584338 8850 tablet_bootstrap.cc:492] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: No bootstrap required, opened a new log
I20260619 12:05:09.584301 8864 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.584374 8864 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:05:09.584406 8850 ts_tablet_manager.cc:1397] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:09.584566 8850 raft_consensus.cc:348] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.584628 8850 raft_consensus.cc:374] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.584643 8850 raft_consensus.cc:729] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e115141793984654a21f10d69f29fdc8, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.584693 8850 consensus_queue.cc:260] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.584791 8850 ts_tablet_manager.cc:1428] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:09.584854 8853 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.584928 8853 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:05:09.602628 8864 raft_consensus.cc:482] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:09.602743 8864 raft_consensus.cc:504] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.602921 8864 leader_election.cc:288] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.603230 8771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5baf37a5893455e8f267637271fbb11" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8" is_pre_election: true
I20260619 12:05:09.603339 8771 raft_consensus.cc:2454] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 0.
I20260619 12:05:09.603260 8631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5baf37a5893455e8f267637271fbb11" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86b849dc6c74d3590cf97db8f63986b" is_pre_election: true
I20260619 12:05:09.603521 8631 raft_consensus.cc:2454] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 0.
I20260619 12:05:09.603582 8422 leader_election.cc:302] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31cb1552f79c4ac3847c6d0c3695784d, e115141793984654a21f10d69f29fdc8; no voters:
I20260619 12:05:09.603709 8864 raft_consensus.cc:2781] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:09.603749 8864 raft_consensus.cc:482] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:09.603770 8864 raft_consensus.cc:3037] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.604570 8864 raft_consensus.cc:504] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.604732 8864 leader_election.cc:288] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 election: Requested vote from peers e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.604921 8631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5baf37a5893455e8f267637271fbb11" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86b849dc6c74d3590cf97db8f63986b"
I20260619 12:05:09.605019 8631 raft_consensus.cc:3037] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.604971 8771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5baf37a5893455e8f267637271fbb11" candidate_uuid: "31cb1552f79c4ac3847c6d0c3695784d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8"
I20260619 12:05:09.605063 8771 raft_consensus.cc:3037] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.605854 8631 raft_consensus.cc:2454] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 1.
I20260619 12:05:09.605880 8771 raft_consensus.cc:2454] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31cb1552f79c4ac3847c6d0c3695784d in term 1.
I20260619 12:05:09.606122 8422 leader_election.cc:302] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31cb1552f79c4ac3847c6d0c3695784d, e115141793984654a21f10d69f29fdc8; no voters:
I20260619 12:05:09.606241 8864 raft_consensus.cc:2781] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:09.606319 8864 raft_consensus.cc:686] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 LEADER]: Becoming Leader. State: Replica: 31cb1552f79c4ac3847c6d0c3695784d, State: Running, Role: LEADER
I20260619 12:05:09.606397 8864 consensus_queue.cc:237] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.606839 8851 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: TxnStatusTablet state changed. Reason: New leader 31cb1552f79c4ac3847c6d0c3695784d. Latest consensus state: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.606943 8851 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:05:09.607259 8890 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:05:09.607462 8890 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:05:09.607470 8332 catalog_manager.cc:5495] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d reported cstate change: term changed from 0 to 1, leader changed from <none> to 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129). New cstate: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } health_report { overall_health: UNKNOWN } } }
I20260619 12:05:09.616046 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.613093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.0.203.129:41039)
0619 12:05:09.613170 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.613173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.613183 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:09.613208 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.613209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.613219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.613273 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.613416 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.613944 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.615188 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.615453 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.615517 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.615754 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.615797 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.615856 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.615946 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.615976 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:05:09.616132 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.612802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41039 (local address 127.0.0.1:38942)
0619 12:05:09.612886 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.612898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.612945 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.613296 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.613297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.613299 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:05:09.613357 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.613362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.614046 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.614048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.615082 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.615086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.615528 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.615529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.615602 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.615630 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.615868 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.615885 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.615942 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.616012 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 12:05:09.617395 8631 raft_consensus.cc:1261] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Refusing update from remote peer 31cb1552f79c4ac3847c6d0c3695784d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.617492 8771 raft_consensus.cc:1261] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Refusing update from remote peer 31cb1552f79c4ac3847c6d0c3695784d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.617640 8864 consensus_queue.cc:1035] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.617791 8864 consensus_queue.cc:1035] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.618206 8849 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: TxnStatusTablet state changed. Reason: New leader 31cb1552f79c4ac3847c6d0c3695784d. Latest consensus state: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.618316 8849 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:05:09.618350 8853 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: TxnStatusTablet state changed. Reason: New leader 31cb1552f79c4ac3847c6d0c3695784d. Latest consensus state: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.618453 8853 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:05:09.619012 8864 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.619095 8864 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:05:09.619299 8859 mvcc.cc:205] Tried to move back new op lower bound from 7298542426590519296 to 7298542426548183040. Current Snapshot: MvccSnapshot[applied={T|T < 7298542426590519296 or (T in {7298542426590519296})}]
I20260619 12:05:09.619570 8849 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.619652 8849 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P e86b849dc6c74d3590cf97db8f63986b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:05:09.623085 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.620928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.0.203.130:34725)
0619 12:05:09.620975 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.620978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.620996 (+ 18us) server_negotiation.cc:407] Connection header received
0619 12:05:09.621033 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.621034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.621056 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.621114 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.621217 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.621703 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.622385 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.622578 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.622632 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.622764 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.622880 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.622941 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.622988 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.623010 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:05:09.628631 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.625985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41039 (local address 127.0.203.130:50219)
0619 12:05:09.626177 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.626187 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.626245 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.626377 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.626378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.626380 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:05:09.626457 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.626458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.627071 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.627072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.627812 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.627815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.628218 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.628219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.628289 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.628304 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.628460 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.628462 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.628502 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.628542 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260619 12:05:09.628609 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.626053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:50219 (local address 127.0.203.129:41039)
0619 12:05:09.626102 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.626104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.626247 (+ 143us) server_negotiation.cc:407] Connection header received
0619 12:05:09.626305 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.626306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.626319 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.626373 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.626511 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.627007 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.627924 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.628157 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.628211 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.628380 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.628414 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.628456 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.628505 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.628527 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:05:09.629496 8864 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.629593 8864 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P 31cb1552f79c4ac3847c6d0c3695784d: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:05:09.630293 8853 tablet_replica.cc:439] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31cb1552f79c4ac3847c6d0c3695784d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } } }
I20260619 12:05:09.630407 8853 tablet_replica.cc:442] T a5baf37a5893455e8f267637271fbb11 P e115141793984654a21f10d69f29fdc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:05:09.631910 8771 raft_consensus.cc:1261] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Refusing update from remote peer e86b849dc6c74d3590cf97db8f63986b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.632006 8491 raft_consensus.cc:1261] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Refusing update from remote peer e86b849dc6c74d3590cf97db8f63986b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.632138 8877 consensus_queue.cc:1035] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.632273 8849 consensus_queue.cc:1035] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [LEADER]: Connected to new peer: Peer: permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.641490 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.638484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:44155 (local address 127.0.203.130:34725)
0619 12:05:09.638529 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.638532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.638664 (+ 132us) server_negotiation.cc:407] Connection header received
0619 12:05:09.638732 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.638733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.638741 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.638794 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.638945 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.639549 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.640771 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.640989 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.641056 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.641262 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.641300 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.641352 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.641409 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.641427 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:05:09.641563 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.638414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34725 (local address 127.0.203.129:44155)
0619 12:05:09.638593 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.638606 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.638676 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.638799 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.638800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.638802 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:05:09.638876 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.638878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.639624 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.639625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.640686 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.640689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.641061 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.641062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.641126 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.641146 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.641355 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.641357 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.641397 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.641454 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260619 12:05:09.650565 8333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.650722 8333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.651503 8333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Sending DeleteTablet for 3 replicas of tablet 8a5fe501375e4cdb956f693029dbeeeb
I20260619 12:05:09.651840 8471 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5fe501375e4cdb956f693029dbeeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:38916
I20260619 12:05:09.651880 8611 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5fe501375e4cdb956f693029dbeeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:59204
I20260619 12:05:09.651998 8869 tablet_replica.cc:330] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: stopping tablet replica
I20260619 12:05:09.652017 8751 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5fe501375e4cdb956f693029dbeeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:58196
I20260619 12:05:09.652076 8869 raft_consensus.cc:2229] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:09.652117 8869 raft_consensus.cc:2258] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.652127 8871 tablet_replica.cc:330] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: stopping tablet replica
I20260619 12:05:09.652190 8870 tablet_replica.cc:330] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: stopping tablet replica
I20260619 12:05:09.652206 8871 raft_consensus.cc:2229] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:09.652257 8871 raft_consensus.cc:2258] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.652259 8870 raft_consensus.cc:2229] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:05:09.652352 8870 raft_consensus.cc:2258] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.652953 8870 ts_tablet_manager.cc:1905] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.652988 8869 ts_tablet_manager.cc:1905] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.654791 8870 ts_tablet_manager.cc:1918] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:05:09.654846 8870 log.cc:1196] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/8a5fe501375e4cdb956f693029dbeeeb
I20260619 12:05:09.654989 8869 ts_tablet_manager.cc:1918] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:05:09.655047 8869 log.cc:1196] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/8a5fe501375e4cdb956f693029dbeeeb
I20260619 12:05:09.655066 8870 ts_tablet_manager.cc:1939] T 8a5fe501375e4cdb956f693029dbeeeb P e86b849dc6c74d3590cf97db8f63986b: Deleting consensus metadata
I20260619 12:05:09.655275 8869 ts_tablet_manager.cc:1939] T 8a5fe501375e4cdb956f693029dbeeeb P 31cb1552f79c4ac3847c6d0c3695784d: Deleting consensus metadata
I20260619 12:05:09.655486 8319 catalog_manager.cc:4841] TS e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725): tablet 8a5fe501375e4cdb956f693029dbeeeb (table test-table [id=04e2b7664852477fa932e679d09676ff]) successfully deleted
I20260619 12:05:09.655565 8871 ts_tablet_manager.cc:1905] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.655850 8319 catalog_manager.cc:4841] TS 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039): tablet 8a5fe501375e4cdb956f693029dbeeeb (table test-table [id=04e2b7664852477fa932e679d09676ff]) successfully deleted
I20260619 12:05:09.656893 8871 ts_tablet_manager.cc:1918] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:05:09.656960 8871 log.cc:1196] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/8a5fe501375e4cdb956f693029dbeeeb
I20260619 12:05:09.657155 8871 ts_tablet_manager.cc:1939] T 8a5fe501375e4cdb956f693029dbeeeb P e115141793984654a21f10d69f29fdc8: Deleting consensus metadata
I20260619 12:05:09.657558 8318 catalog_manager.cc:4841] TS e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001): tablet 8a5fe501375e4cdb956f693029dbeeeb (table test-table [id=04e2b7664852477fa932e679d09676ff]) successfully deleted
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:05:09.690435 8885 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:09.682233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.0.203.190:42303)
0619 12:05:09.682287 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.682289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.682459 (+ 170us) server_negotiation.cc:407] Connection header received
0619 12:05:09.682569 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.682570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.682585 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.682655 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:09.683531 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.684379 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.685627 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.685987 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.686074 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.690346 (+ 4272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60302: BlockingRecv error: recv got EOF from 127.0.0.1:60302 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM
I20260619 12:05:09.702033 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.691825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.0.203.190:42303)
0619 12:05:09.691872 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.691874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.691957 (+ 83us) server_negotiation.cc:407] Connection header received
0619 12:05:09.692041 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.692042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.692060 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.692123 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:09.692768 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.693554 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.694676 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.694990 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.695077 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.698299 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:09.698300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:09.698301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:09.698309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:09.700201 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.700828 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.700830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.700830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.701087 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:09.701326 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:09.701328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:09.701328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:09.701548 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:09.701669 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.701817 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.701919 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:05:09.704396 8333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:05:09.704627 8333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:09.706321 8471 tablet_service.cc:1467] Processing CreateTablet for tablet eb78cee6170540bdb99ffc266404acf5 (DEFAULT_TABLE table=test-table [id=9a78586551354f02ba54cf9087f0c05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.706373 8751 tablet_service.cc:1467] Processing CreateTablet for tablet eb78cee6170540bdb99ffc266404acf5 (DEFAULT_TABLE table=test-table [id=9a78586551354f02ba54cf9087f0c05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.706529 8751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb78cee6170540bdb99ffc266404acf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.706524 8471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb78cee6170540bdb99ffc266404acf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.707557 8611 tablet_service.cc:1467] Processing CreateTablet for tablet eb78cee6170540bdb99ffc266404acf5 (DEFAULT_TABLE table=test-table [id=9a78586551354f02ba54cf9087f0c05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.707688 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb78cee6170540bdb99ffc266404acf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.708393 8846 tablet_bootstrap.cc:492] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: Bootstrap starting.
I20260619 12:05:09.708801 8846 tablet_bootstrap.cc:654] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.709031 8845 tablet_bootstrap.cc:492] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: Bootstrap starting.
I20260619 12:05:09.709633 8845 tablet_bootstrap.cc:654] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.710414 8845 tablet_bootstrap.cc:492] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: No bootstrap required, opened a new log
I20260619 12:05:09.710573 8845 ts_tablet_manager.cc:1397] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:05:09.710793 8845 raft_consensus.cc:348] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.710883 8845 raft_consensus.cc:374] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.710903 8845 raft_consensus.cc:729] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86b849dc6c74d3590cf97db8f63986b, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.710952 8845 consensus_queue.cc:260] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.711050 8845 ts_tablet_manager.cc:1428] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:05:09.712195 8850 tablet_bootstrap.cc:492] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: Bootstrap starting.
I20260619 12:05:09.712709 8850 tablet_bootstrap.cc:654] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.712761 8846 tablet_bootstrap.cc:492] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: No bootstrap required, opened a new log
I20260619 12:05:09.712823 8846 ts_tablet_manager.cc:1397] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:05:09.712985 8846 raft_consensus.cc:348] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.713058 8846 raft_consensus.cc:374] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.713078 8846 raft_consensus.cc:729] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cb1552f79c4ac3847c6d0c3695784d, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.713135 8846 consensus_queue.cc:260] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.713234 8846 ts_tablet_manager.cc:1428] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:05:09.715814 8850 tablet_bootstrap.cc:492] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: No bootstrap required, opened a new log
I20260619 12:05:09.715898 8850 ts_tablet_manager.cc:1397] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:05:09.716097 8850 raft_consensus.cc:348] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.716212 8850 raft_consensus.cc:374] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.716233 8850 raft_consensus.cc:729] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e115141793984654a21f10d69f29fdc8, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.716290 8850 consensus_queue.cc:260] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.716398 8850 ts_tablet_manager.cc:1428] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:09.753388 8903 raft_consensus.cc:482] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:09.753525 8903 raft_consensus.cc:504] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.753721 8903 leader_election.cc:288] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.754112 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb78cee6170540bdb99ffc266404acf5" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8" is_pre_election: true
I20260619 12:05:09.754230 8770 raft_consensus.cc:2454] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 0.
I20260619 12:05:09.754153 8491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb78cee6170540bdb99ffc266404acf5" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cb1552f79c4ac3847c6d0c3695784d" is_pre_election: true
I20260619 12:05:09.754313 8491 raft_consensus.cc:2454] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 0.
I20260619 12:05:09.754395 8562 leader_election.cc:302] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e115141793984654a21f10d69f29fdc8, e86b849dc6c74d3590cf97db8f63986b; no voters:
I20260619 12:05:09.754565 8903 raft_consensus.cc:2781] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:09.754609 8903 raft_consensus.cc:482] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:09.754631 8903 raft_consensus.cc:3037] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.755633 8903 raft_consensus.cc:504] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.755810 8903 leader_election.cc:288] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 election: Requested vote from peers 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039), e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001)
I20260619 12:05:09.756110 8770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb78cee6170540bdb99ffc266404acf5" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e115141793984654a21f10d69f29fdc8"
I20260619 12:05:09.756222 8770 raft_consensus.cc:3037] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.756138 8491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb78cee6170540bdb99ffc266404acf5" candidate_uuid: "e86b849dc6c74d3590cf97db8f63986b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cb1552f79c4ac3847c6d0c3695784d"
I20260619 12:05:09.756345 8491 raft_consensus.cc:3037] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.757251 8770 raft_consensus.cc:2454] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 1.
I20260619 12:05:09.757352 8491 raft_consensus.cc:2454] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86b849dc6c74d3590cf97db8f63986b in term 1.
I20260619 12:05:09.757495 8562 leader_election.cc:302] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e115141793984654a21f10d69f29fdc8, e86b849dc6c74d3590cf97db8f63986b; no voters:
I20260619 12:05:09.757638 8903 raft_consensus.cc:2781] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:09.757716 8903 raft_consensus.cc:686] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 1 LEADER]: Becoming Leader. State: Replica: e86b849dc6c74d3590cf97db8f63986b, State: Running, Role: LEADER
I20260619 12:05:09.757783 8903 consensus_queue.cc:237] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.758666 8333 catalog_manager.cc:5495] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b reported cstate change: term changed from 0 to 1, leader changed from <none> to e86b849dc6c74d3590cf97db8f63986b (127.0.203.130). New cstate: current_term: 1 leader_uuid: "e86b849dc6c74d3590cf97db8f63986b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } health_report { overall_health: UNKNOWN } } }
I20260619 12:05:09.772310 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.769356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.0.203.130:34725)
0619 12:05:09.769438 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.769440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.769452 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:05:09.769478 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.769479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.769488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.769549 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.769738 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.770279 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.771510 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.771767 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.771847 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.772042 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.772078 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.772124 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.772202 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.772222 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:05:09.773370 8491 raft_consensus.cc:1261] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Refusing update from remote peer e86b849dc6c74d3590cf97db8f63986b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.773370 8770 raft_consensus.cc:1261] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Refusing update from remote peer e86b849dc6c74d3590cf97db8f63986b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.773698 8903 consensus_queue.cc:1035] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [LEADER]: Connected to new peer: Peer: permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.773842 8877 consensus_queue.cc:1035] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.777053 8333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.777186 8333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.778000 8333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Sending DeleteTablet for 3 replicas of tablet eb78cee6170540bdb99ffc266404acf5
I20260619 12:05:09.778280 8471 tablet_service.cc:1514] Processing DeleteTablet for tablet eb78cee6170540bdb99ffc266404acf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:38916
I20260619 12:05:09.778393 8869 tablet_replica.cc:330] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: stopping tablet replica
I20260619 12:05:09.778455 8869 raft_consensus.cc:2229] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:09.778393 8611 tablet_service.cc:1514] Processing DeleteTablet for tablet eb78cee6170540bdb99ffc266404acf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:59204
I20260619 12:05:09.778509 8869 raft_consensus.cc:2258] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.778522 8870 tablet_replica.cc:330] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: stopping tablet replica
I20260619 12:05:09.778568 8870 raft_consensus.cc:2229] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:05:09.778643 8870 raft_consensus.cc:2258] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.778868 8333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 12:05:09.779028 8751 tablet_service.cc:1514] Processing DeleteTablet for tablet eb78cee6170540bdb99ffc266404acf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:58196
W20260619 12:05:09.779060 8333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:09.779116 8871 tablet_replica.cc:330] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: stopping tablet replica
I20260619 12:05:09.779170 8871 raft_consensus.cc:2229] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:09.779215 8871 raft_consensus.cc:2258] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.779517 8869 ts_tablet_manager.cc:1905] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.779714 8871 ts_tablet_manager.cc:1905] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.781191 8871 ts_tablet_manager.cc:1918] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:09.781258 8871 log.cc:1196] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/eb78cee6170540bdb99ffc266404acf5
I20260619 12:05:09.781347 8869 ts_tablet_manager.cc:1918] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:09.781423 8869 log.cc:1196] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/eb78cee6170540bdb99ffc266404acf5
I20260619 12:05:09.781476 8871 ts_tablet_manager.cc:1939] T eb78cee6170540bdb99ffc266404acf5 P e115141793984654a21f10d69f29fdc8: Deleting consensus metadata
I20260619 12:05:09.781672 8869 ts_tablet_manager.cc:1939] T eb78cee6170540bdb99ffc266404acf5 P 31cb1552f79c4ac3847c6d0c3695784d: Deleting consensus metadata
I20260619 12:05:09.782044 8319 catalog_manager.cc:4841] TS 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039): tablet eb78cee6170540bdb99ffc266404acf5 (table test-table [id=9a78586551354f02ba54cf9087f0c05e]) successfully deleted
I20260619 12:05:09.782354 8318 catalog_manager.cc:4841] TS e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001): tablet eb78cee6170540bdb99ffc266404acf5 (table test-table [id=9a78586551354f02ba54cf9087f0c05e]) successfully deleted
I20260619 12:05:09.782689 8870 ts_tablet_manager.cc:1905] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.782860 8751 tablet_service.cc:1467] Processing CreateTablet for tablet dc940f7ad7944a959557f44653d79c5a (DEFAULT_TABLE table=test-table [id=7b96f16dc7454f6f95808e799a5597c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.782984 8751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc940f7ad7944a959557f44653d79c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.784109 8870 ts_tablet_manager.cc:1918] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:09.784157 8870 log.cc:1196] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/eb78cee6170540bdb99ffc266404acf5
I20260619 12:05:09.784267 8850 tablet_bootstrap.cc:492] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: Bootstrap starting.
I20260619 12:05:09.784293 8611 tablet_service.cc:1467] Processing CreateTablet for tablet dc940f7ad7944a959557f44653d79c5a (DEFAULT_TABLE table=test-table [id=7b96f16dc7454f6f95808e799a5597c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.784337 8870 ts_tablet_manager.cc:1939] T eb78cee6170540bdb99ffc266404acf5 P e86b849dc6c74d3590cf97db8f63986b: Deleting consensus metadata
I20260619 12:05:09.784411 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc940f7ad7944a959557f44653d79c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.784521 8471 tablet_service.cc:1467] Processing CreateTablet for tablet dc940f7ad7944a959557f44653d79c5a (DEFAULT_TABLE table=test-table [id=7b96f16dc7454f6f95808e799a5597c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:09.784649 8471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc940f7ad7944a959557f44653d79c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:09.784996 8850 tablet_bootstrap.cc:654] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.785701 8845 tablet_bootstrap.cc:492] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: Bootstrap starting.
I20260619 12:05:09.786007 8845 tablet_bootstrap.cc:654] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.786255 8846 tablet_bootstrap.cc:492] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d: Bootstrap starting.
I20260619 12:05:09.786525 8319 catalog_manager.cc:4841] TS e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725): tablet eb78cee6170540bdb99ffc266404acf5 (table test-table [id=9a78586551354f02ba54cf9087f0c05e]) successfully deleted
I20260619 12:05:09.786597 8846 tablet_bootstrap.cc:654] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:09.786934 8850 tablet_bootstrap.cc:492] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: No bootstrap required, opened a new log
I20260619 12:05:09.786988 8850 ts_tablet_manager.cc:1397] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:05:09.787123 8846 tablet_bootstrap.cc:492] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d: No bootstrap required, opened a new log
I20260619 12:05:09.787165 8846 ts_tablet_manager.cc:1397] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:09.787180 8850 raft_consensus.cc:348] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.787261 8850 raft_consensus.cc:374] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.787278 8850 raft_consensus.cc:729] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e115141793984654a21f10d69f29fdc8, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.787281 8846 raft_consensus.cc:348] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.787343 8846 raft_consensus.cc:374] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.787324 8850 consensus_queue.cc:260] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.787396 8846 raft_consensus.cc:729] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cb1552f79c4ac3847c6d0c3695784d, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.787389 8845 tablet_bootstrap.cc:492] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: No bootstrap required, opened a new log
I20260619 12:05:09.787441 8845 ts_tablet_manager.cc:1397] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:05:09.787438 8846 consensus_queue.cc:260] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.787467 8850 ts_tablet_manager.cc:1428] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:09.787528 8846 ts_tablet_manager.cc:1428] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:05:09.787628 8845 raft_consensus.cc:348] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.787711 8845 raft_consensus.cc:374] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:09.787739 8845 raft_consensus.cc:729] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86b849dc6c74d3590cf97db8f63986b, State: Initialized, Role: FOLLOWER
I20260619 12:05:09.787788 8845 consensus_queue.cc:260] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.787894 8845 ts_tablet_manager.cc:1428] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:09.789856 8853 raft_consensus.cc:482] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:09.789942 8853 raft_consensus.cc:504] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.790267 8853 leader_election.cc:288] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039), e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725)
I20260619 12:05:09.793776 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.790808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:55497 (local address 127.0.203.130:34725)
0619 12:05:09.790854 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.790856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.790983 (+ 127us) server_negotiation.cc:407] Connection header received
0619 12:05:09.791088 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.791089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.791104 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.791132 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.791423 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.791943 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.793023 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.793401 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.793473 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.793551 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.793653 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.793695 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:05:09.793808 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.790695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34725 (local address 127.0.203.131:55497)
0619 12:05:09.790908 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.790920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.791012 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.791202 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.791203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.791206 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.791295 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.791297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.792054 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.792055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.792928 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.792930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.793506 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.793507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.793577 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.793583 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.793647 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.793703 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":47,"threads_started":1}
I20260619 12:05:09.794098 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.790572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:48987 (local address 127.0.203.129:41039)
0619 12:05:09.790639 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.790641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.790796 (+ 155us) server_negotiation.cc:407] Connection header received
0619 12:05:09.790896 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.790897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.790906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.790933 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:09.791178 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.791881 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.793024 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.793416 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.793499 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.793513 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.793802 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.794019 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:05:09.794396 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.790473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41039 (local address 127.0.203.131:48987)
0619 12:05:09.790706 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.790718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.790826 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.790999 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.791000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.791003 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:09.791076 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.791078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.791965 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.791967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.792925 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.792928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.793579 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.793580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.793674 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.793680 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.794222 (+ 542us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.794289 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":15,"thread_start_us":114,"threads_started":1}
I20260619 12:05:09.794404 8631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc940f7ad7944a959557f44653d79c5a" candidate_uuid: "e115141793984654a21f10d69f29fdc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86b849dc6c74d3590cf97db8f63986b" is_pre_election: true
I20260619 12:05:09.794540 8631 raft_consensus.cc:2454] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e115141793984654a21f10d69f29fdc8 in term 0.
I20260619 12:05:09.794719 8703 leader_election.cc:302] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e115141793984654a21f10d69f29fdc8, e86b849dc6c74d3590cf97db8f63986b; no voters:
I20260619 12:05:09.794715 8491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc940f7ad7944a959557f44653d79c5a" candidate_uuid: "e115141793984654a21f10d69f29fdc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cb1552f79c4ac3847c6d0c3695784d" is_pre_election: true
I20260619 12:05:09.794804 8491 raft_consensus.cc:2454] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e115141793984654a21f10d69f29fdc8 in term 0.
I20260619 12:05:09.794870 8853 raft_consensus.cc:2781] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:09.794911 8853 raft_consensus.cc:482] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:09.794930 8853 raft_consensus.cc:3037] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.795761 8853 raft_consensus.cc:504] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.795930 8853 leader_election.cc:288] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [CANDIDATE]: Term 1 election: Requested vote from peers 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039), e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725)
I20260619 12:05:09.796132 8491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc940f7ad7944a959557f44653d79c5a" candidate_uuid: "e115141793984654a21f10d69f29fdc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31cb1552f79c4ac3847c6d0c3695784d"
I20260619 12:05:09.796227 8491 raft_consensus.cc:3037] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.796183 8631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc940f7ad7944a959557f44653d79c5a" candidate_uuid: "e115141793984654a21f10d69f29fdc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86b849dc6c74d3590cf97db8f63986b"
I20260619 12:05:09.796267 8631 raft_consensus.cc:3037] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:09.796934 8631 raft_consensus.cc:2454] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e115141793984654a21f10d69f29fdc8 in term 1.
I20260619 12:05:09.797040 8491 raft_consensus.cc:2454] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e115141793984654a21f10d69f29fdc8 in term 1.
I20260619 12:05:09.797202 8703 leader_election.cc:302] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e115141793984654a21f10d69f29fdc8, e86b849dc6c74d3590cf97db8f63986b; no voters:
I20260619 12:05:09.797358 8853 raft_consensus.cc:2781] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:09.797618 8853 raft_consensus.cc:686] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 1 LEADER]: Becoming Leader. State: Replica: e115141793984654a21f10d69f29fdc8, State: Running, Role: LEADER
I20260619 12:05:09.797710 8853 consensus_queue.cc:237] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } }
I20260619 12:05:09.798687 8332 catalog_manager.cc:5495] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e115141793984654a21f10d69f29fdc8 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "e115141793984654a21f10d69f29fdc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e115141793984654a21f10d69f29fdc8" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 40001 } health_report { overall_health: HEALTHY } } }
I20260619 12:05:09.809384 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.806761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.0.203.131:40001)
0619 12:05:09.806804 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.806806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.806841 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:05:09.806917 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.806918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.806936 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.807005 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.807190 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.807828 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.808685 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.808885 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.808957 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.809097 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.809213 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.809267 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.809304 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.809320 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:05:09.814953 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.812486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:58515 (local address 127.0.203.129:41039)
0619 12:05:09.812543 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.812545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.812693 (+ 148us) server_negotiation.cc:407] Connection header received
0619 12:05:09.812723 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.812724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.812734 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.812782 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.812911 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.813399 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.814231 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.814451 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.814516 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.814711 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.814748 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.814817 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.814859 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.814881 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:05:09.814980 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.812383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41039 (local address 127.0.203.131:58515)
0619 12:05:09.812539 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.812587 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.812649 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.812794 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.812794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.812796 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:05:09.812863 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.812864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.813473 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.813474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.814128 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.814129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.814519 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.814520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.814585 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.814603 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.814821 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.814822 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.814845 (+ 23us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.814891 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260619 12:05:09.817516 8631 raft_consensus.cc:1261] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Refusing update from remote peer e115141793984654a21f10d69f29fdc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.817566 8491 raft_consensus.cc:1261] T dc940f7ad7944a959557f44653d79c5a P 31cb1552f79c4ac3847c6d0c3695784d [term 1 FOLLOWER]: Refusing update from remote peer e115141793984654a21f10d69f29fdc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:09.817843 8929 consensus_queue.cc:1035] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31cb1552f79c4ac3847c6d0c3695784d" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.817991 8853 consensus_queue.cc:1035] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e86b849dc6c74d3590cf97db8f63986b" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:09.826683 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.823849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:53807 (local address 127.0.203.131:40001)
0619 12:05:09.824037 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:09.824040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:09.824050 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:09.824074 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:09.824074 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:09.824086 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:09.824141 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:09.824351 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.824831 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.825775 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.826029 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.826104 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.826355 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.826398 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.826451 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:09.826570 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 12:05:09.826610 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260619 12:05:09.826728 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.823808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:40001 (local address 127.0.203.129:53807)
0619 12:05:09.823864 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.823878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:09.823927 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:09.824201 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:09.824202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:09.824204 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:05:09.824267 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.824270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.824912 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.824913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.825664 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:09.825668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:09.826145 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:09.826148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:09.826253 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:09.826265 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:05:09.826480 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:05:09.826483 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:05:09.826526 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 12:05:09.826632 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 12:05:09.838294 8332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.838435 8332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:09.839279 8332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 153d3e25c49d4e8da669c814e6640fb8: Sending DeleteTablet for 3 replicas of tablet dc940f7ad7944a959557f44653d79c5a
I20260619 12:05:09.839627 8751 tablet_service.cc:1514] Processing DeleteTablet for tablet dc940f7ad7944a959557f44653d79c5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:58196
I20260619 12:05:09.839650 8611 tablet_service.cc:1514] Processing DeleteTablet for tablet dc940f7ad7944a959557f44653d79c5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:09 UTC) from {username='oryx', principal='oryx/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:59204
I20260619 12:05:09.839772 8870 tablet_replica.cc:330] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: stopping tablet replica
I20260619 12:05:09.839792 8871 tablet_replica.cc:330] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: stopping tablet replica
I20260619 12:05:09.839829 8870 raft_consensus.cc:2229] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:09.839854 8871 raft_consensus.cc:2229] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:05:09.839871 8870 raft_consensus.cc:2258] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.839964 8871 raft_consensus.cc:2258] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:09.840315 8870 ts_tablet_manager.cc:1905] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.840479 8871 ts_tablet_manager.cc:1905] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:09.840500 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 8406
I20260619 12:05:09.842973 8871 ts_tablet_manager.cc:1918] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:05:09.843022 8870 ts_tablet_manager.cc:1918] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:05:09.843037 8871 log.cc:1196] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/dc940f7ad7944a959557f44653d79c5a
I20260619 12:05:09.843067 8870 log.cc:1196] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/dc940f7ad7944a959557f44653d79c5a
I20260619 12:05:09.843261 8871 ts_tablet_manager.cc:1939] T dc940f7ad7944a959557f44653d79c5a P e115141793984654a21f10d69f29fdc8: Deleting consensus metadata
I20260619 12:05:09.843261 8870 ts_tablet_manager.cc:1939] T dc940f7ad7944a959557f44653d79c5a P e86b849dc6c74d3590cf97db8f63986b: Deleting consensus metadata
I20260619 12:05:09.843684 8319 catalog_manager.cc:4841] TS e86b849dc6c74d3590cf97db8f63986b (127.0.203.130:34725): tablet dc940f7ad7944a959557f44653d79c5a (table test-table [id=7b96f16dc7454f6f95808e799a5597c8]) successfully deleted
I20260619 12:05:09.843684 8318 catalog_manager.cc:4841] TS e115141793984654a21f10d69f29fdc8 (127.0.203.131:40001): tablet dc940f7ad7944a959557f44653d79c5a (table test-table [id=7b96f16dc7454f6f95808e799a5597c8]) successfully deleted
W20260619 12:05:09.848738 8319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:41039 (error 108)
I20260619 12:05:09.849184 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 8546
I20260619 12:05:09.850062 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:09.849297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41039 (local address 127.0.0.1:38952)
0619 12:05:09.849956 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:09.850002 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:41039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":109,"threads_started":1}
W20260619 12:05:09.850175 8319 catalog_manager.cc:4584] TS 31cb1552f79c4ac3847c6d0c3695784d (127.0.203.129:41039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc940f7ad7944a959557f44653d79c5a: Network error: Client connection negotiation failed: client connection to 127.0.203.129:41039: connect: Connection refused (error 111)
I20260619 12:05:09.855964 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 8686
I20260619 12:05:09.861456 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 8302
2026-06-19T12:05:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2656 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8942](info): set up 2 sockets
Jun 19 12:05:09 dist-test-slave-44vj krb5kdc[8942](info): commencing operation
krb5kdc: starting...
W20260619 12:05:10.832338 814 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:05:10 dist-test-slave-44vj krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:05:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:05:10.962147 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38953
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:38953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:11.053421 8955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:11.053725 8955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:11.053763 8955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:11.055141 8955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:11.055197 8955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:11.055217 8955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:11.055233 8955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:11.055248 8955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:11.057119 8955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:38953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:11.057428 8955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:11.057682 8955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:11.063174 8961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:11.063186 8960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:11.063277 8955 server_base.cc:1034] running on GCE node
W20260619 12:05:11.063140 8963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:11.063709 8955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:11.064069 8955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:11.065254 8955 hybrid_clock.cc:648] HybridClock initialized: now 1781870711065231 us; error 56 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T12:05:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1208 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:05:11 dist-test-slave-44vj krb5kdc[8968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:05:11 dist-test-slave-44vj krb5kdc[8968](info): set up 2 sockets
Jun 19 12:05:11 dist-test-slave-44vj krb5kdc[8968](info): commencing operation
krb5kdc: starting...
W20260619 12:05:12.046924 814 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:05:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:05:12.183882 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33023
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:12.276042 8981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:12.276314 8981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:12.276340 8981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:12.277714 8981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:12.277761 8981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:12.277774 8981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:12.277786 8981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:12.277797 8981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:12.279706 8981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:12.280014 8981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:12.280277 8981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:12.286425 8986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:12.286398 8987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:12.286582 8981 server_base.cc:1034] running on GCE node
W20260619 12:05:12.286403 8989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:12.287186 8981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:12.287658 8981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:12.288863 8981 hybrid_clock.cc:648] HybridClock initialized: now 1781870712288828 us; error 68 us; skew 500 ppm
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:12.295058 8981 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:05:12.295785 8981 webserver.cc:460] Webserver started at http://127.0.203.190:34027/ using document root <none> and password file <none>
I20260619 12:05:12.296067 8981 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:12.296131 8981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:12.296283 8981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:12.297571 8981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "5070c32bd7f74a59a26c7ea01de30bdf"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "242ae91030034c6d73b315cfcecc406f"
server_key_iv: "7b10ceab2739212d9d814248f01dc080"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.297966 8981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "5070c32bd7f74a59a26c7ea01de30bdf"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "242ae91030034c6d73b315cfcecc406f"
server_key_iv: "7b10ceab2739212d9d814248f01dc080"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.299739 8981 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:05:12.300933 8996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.301241 8981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:12.301362 8981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "5070c32bd7f74a59a26c7ea01de30bdf"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "242ae91030034c6d73b315cfcecc406f"
server_key_iv: "7b10ceab2739212d9d814248f01dc080"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.301446 8981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:12.321765 8981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:12.324599 8981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:12.324784 8981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:12.329928 8981 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:33023
I20260619 12:05:12.330050 9048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:33023 every 8 connection(s)
I20260619 12:05:12.330575 8981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:05:12.331475 9049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:12.333918 9049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Bootstrap starting.
I20260619 12:05:12.334568 9049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:12.334868 9049 log.cc:824] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:12.335870 9049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: No bootstrap required, opened a new log
I20260619 12:05:12.337546 9049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:12.337706 9049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:12.337728 9049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5070c32bd7f74a59a26c7ea01de30bdf, State: Initialized, Role: FOLLOWER
I20260619 12:05:12.337865 9049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:12.337944 9049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:05:12.337990 9049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:05:12.338047 9049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:12.339041 9049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:12.339140 9049 leader_election.cc:302] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5070c32bd7f74a59a26c7ea01de30bdf; no voters:
I20260619 12:05:12.339315 9049 leader_election.cc:288] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:05:12.339473 9052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:12.339649 9049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:12.339708 9052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 LEADER]: Becoming Leader. State: Replica: 5070c32bd7f74a59a26c7ea01de30bdf, State: Running, Role: LEADER
I20260619 12:05:12.339864 9052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:12.340430 9053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } } }
I20260619 12:05:12.340435 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 8981
I20260619 12:05:12.340524 9053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:12.340575 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:05:12.340806 814 external_mini_cluster.cc:1428] Setting key 0e00c33a1a29664759993fe5e4e66a45
I20260619 12:05:12.340854 9054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5070c32bd7f74a59a26c7ea01de30bdf. Latest consensus state: current_term: 1 leader_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } } }
I20260619 12:05:12.340914 9054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:12.341336 9061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:12.342051 9061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:12.344162 9061 catalog_manager.cc:1347] Generated new cluster ID: 5bc70b47bc9c4de88343be3d338b6a17
I20260619 12:05:12.344238 9061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:05:12.352443 9061 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:05:12.353201 9061 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:12.355098 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:12.341608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.0.203.190:33023)
0619 12:05:12.341978 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:12.341990 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:12.342026 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:05:12.342208 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:12.342218 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:12.342288 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:12.342447 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:12.343301 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.345218 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.346439 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.347205 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.347333 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:12.350629 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:12.350637 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:12.350637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:12.350657 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:12.353024 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.353556 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.353560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.353561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.353695 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.353930 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.353931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.353931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.354234 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:12.354384 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:12.354641 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 12:05:12.354824 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":87,"threads_started":1}
I20260619 12:05:12.357654 9061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Generated new TSK 0
I20260619 12:05:12.357942 9061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:05:12.395788 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33023
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:12.489452 9075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:12.489777 9075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:12.489809 9075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:12.491425 9075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:12.491505 9075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:12.491544 9075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:12.493510 9075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:12.493830 9075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:12.494263 9075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:12.500155 9081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:12.500140 9083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:12.500140 9080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:12.500726 9075 server_base.cc:1034] running on GCE node
I20260619 12:05:12.500998 9075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:12.501317 9075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:12.502514 9075 hybrid_clock.cc:648] HybridClock initialized: now 1781870712502486 us; error 56 us; skew 500 ppm
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:12.505941 9075 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:05:12.506584 9075 webserver.cc:460] Webserver started at http://127.0.203.129:38767/ using document root <none> and password file <none>
I20260619 12:05:12.506865 9075 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:12.506920 9075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:12.507071 9075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:12.508404 9075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "d3f91b7742764e95884b1b6ad4db5f89"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "4a6ef91c8a4a91dfd91944dc57a35222"
server_key_iv: "da69c1c009d389dec4511381eb1e01bb"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.508806 9075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "d3f91b7742764e95884b1b6ad4db5f89"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "4a6ef91c8a4a91dfd91944dc57a35222"
server_key_iv: "da69c1c009d389dec4511381eb1e01bb"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.510620 9075 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 12:05:12.511824 9090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.512132 9075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:12.512254 9075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "d3f91b7742764e95884b1b6ad4db5f89"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "4a6ef91c8a4a91dfd91944dc57a35222"
server_key_iv: "da69c1c009d389dec4511381eb1e01bb"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.512338 9075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:12.529373 9075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:12.533010 9075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:12.533210 9075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:12.533512 9075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:12.534119 9075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:12.534219 9075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.534272 9075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:12.534291 9075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.542285 9075 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:40397
I20260619 12:05:12.542399 9203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:40397 every 8 connection(s)
I20260619 12:05:12.542778 9075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:05:12.552857 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 9075
I20260619 12:05:12.553000 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:05:12.553244 814 external_mini_cluster.cc:1428] Setting key 6044d336a060bbf5f3336ef67d897808
I20260619 12:05:12.556108 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:12.543844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:40143 (local address 127.0.203.190:33023)
0619 12:05:12.543944 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:12.543946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:12.544220 (+ 274us) server_negotiation.cc:407] Connection header received
0619 12:05:12.545295 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:12.545297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:12.545312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:12.545403 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:12.546462 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.547159 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.548980 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.549213 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.549326 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:12.552406 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:12.552413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:12.552413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:12.552423 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:12.554451 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.554992 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.554993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.554994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.555055 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.555321 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.555322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.555322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.555499 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:12.555566 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:12.555882 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 12:05:12.556007 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 12:05:12.556308 9206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:12.543770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:40143)
0619 12:05:12.544000 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:12.544052 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:12.545143 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:12.545535 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:12.545541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:12.545886 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:12.546246 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:12.546254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.547305 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.547306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:12.548832 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:12.548838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.549361 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.549363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:12.549556 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:12.550275 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:12.550286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:12.552225 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:12.554561 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:12.554577 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:12.554581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:12.554908 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:12.555115 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:12.555116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:12.555116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:12.555253 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:12.555563 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:12.555571 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:12.555740 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 12:05:12.555851 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 12:05:12.556065 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":122,"threads_started":1}
I20260619 12:05:12.556808 9204 heartbeater.cc:347] Connected to a master server at 127.0.203.190:33023
I20260619 12:05:12.556910 9204 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:12.557077 9204 heartbeater.cc:511] Master 127.0.203.190:33023 requested a full tablet report, sending...
I20260619 12:05:12.557847 9013 ts_manager.cc:194] Registered new tserver with Master: d3f91b7742764e95884b1b6ad4db5f89 (127.0.203.129:40397)
I20260619 12:05:12.558538 9013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:40143
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:05:12.587890 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33023
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:12.686157 9209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:12.686478 9209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:12.686518 9209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:12.688092 9209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:12.688156 9209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:12.688201 9209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:12.690093 9209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:12.690402 9209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:12.690678 9209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:12.696568 9215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:12.696560 9214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:12.696559 9217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:12.696844 9209 server_base.cc:1034] running on GCE node
I20260619 12:05:12.697078 9209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:12.697347 9209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:12.698547 9209 hybrid_clock.cc:648] HybridClock initialized: now 1781870712698515 us; error 50 us; skew 500 ppm
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:12.702353 9209 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:05:12.703013 9209 webserver.cc:460] Webserver started at http://127.0.203.130:43341/ using document root <none> and password file <none>
I20260619 12:05:12.703329 9209 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:12.703435 9209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:12.703581 9209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:12.704838 9209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "fae0cf6aa41046e7ab5cca079e8c3ef2"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "9590da1952deaa61e7e60f6dc9c32602"
server_key_iv: "688ee4612145721754af6e546a156258"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.705193 9209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "fae0cf6aa41046e7ab5cca079e8c3ef2"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "9590da1952deaa61e7e60f6dc9c32602"
server_key_iv: "688ee4612145721754af6e546a156258"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.706768 9209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:05:12.707839 9224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.708130 9209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:12.708244 9209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "fae0cf6aa41046e7ab5cca079e8c3ef2"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "9590da1952deaa61e7e60f6dc9c32602"
server_key_iv: "688ee4612145721754af6e546a156258"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.708330 9209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:12.722898 9209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:12.725679 9209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:12.725857 9209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:12.726122 9209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:12.726450 9209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:12.726486 9209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.726516 9209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:12.726532 9209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.734107 9209 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:37699
I20260619 12:05:12.734196 9337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:37699 every 8 connection(s)
I20260619 12:05:12.734568 9209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:12.735062 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 9209
I20260619 12:05:12.735195 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:05:12.735457 814 external_mini_cluster.cc:1428] Setting key bfbaf03378f4804bcdcc2547e3e90c28
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
I20260619 12:05:12.748155 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:12.736036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:47479 (local address 127.0.203.190:33023)
0619 12:05:12.736102 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:12.736105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:12.736363 (+ 258us) server_negotiation.cc:407] Connection header received
0619 12:05:12.737842 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:12.737844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:12.737860 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:12.737944 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:12.738977 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.739758 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.741197 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.741570 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.741671 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:12.744545 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:12.744547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:12.744548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:12.744559 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:12.746135 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.746751 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.746753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.746753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.746825 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.747194 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.747196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.747196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.747394 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:12.747483 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:12.747892 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 12:05:12.747994 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:05:12.748474 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:12.735875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:47479)
0619 12:05:12.736153 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:12.736204 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:12.737648 (+ 1444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:12.738069 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:12.738075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:12.738560 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:12.738849 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:12.738856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.739928 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.739929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:12.741082 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:12.741087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.741685 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.741685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:12.741820 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:12.742448 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:12.742458 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:12.744394 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:12.746294 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:12.746309 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:12.746311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:12.746606 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:12.746930 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:12.746931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:12.746931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:12.747070 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:12.747487 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:12.747499 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:12.747731 (+ 232us) client_negotiation.cc:769] Sending connection context
0619 12:05:12.747862 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 12:05:12.748095 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":128,"threads_started":1}
I20260619 12:05:12.749010 9338 heartbeater.cc:347] Connected to a master server at 127.0.203.190:33023
I20260619 12:05:12.749132 9338 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:12.749357 9338 heartbeater.cc:511] Master 127.0.203.190:33023 requested a full tablet report, sending...
I20260619 12:05:12.749788 9013 ts_manager.cc:194] Registered new tserver with Master: fae0cf6aa41046e7ab5cca079e8c3ef2 (127.0.203.130:37699)
I20260619 12:05:12.750213 9013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:47479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:05:12.764251 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33023
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:12.853956 9343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:12.854210 9343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:12.854238 9343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:12.855751 9343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:12.855820 9343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:12.855854 9343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:12.858414 9343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:12.858713 9343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:12.858961 9343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:12.864766 9349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:12.864840 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:12.864840 9351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:12.865332 9343 server_base.cc:1034] running on GCE node
I20260619 12:05:12.865619 9343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:12.865917 9343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:12.867110 9343 hybrid_clock.cc:648] HybridClock initialized: now 1781870712867075 us; error 55 us; skew 500 ppm
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:12.869954 9343 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:05:12.870507 9343 webserver.cc:460] Webserver started at http://127.0.203.131:43873/ using document root <none> and password file <none>
I20260619 12:05:12.870743 9343 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:12.870795 9343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:12.870929 9343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:12.872128 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "66966bc5789b4e2b82e13f2b10a7d454"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "1d88f6cef30fb1e429203f67d4ea2ff0"
server_key_iv: "74ab92a6fa30a7d644aabb7eb8070fb6"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.872489 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "66966bc5789b4e2b82e13f2b10a7d454"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "1d88f6cef30fb1e429203f67d4ea2ff0"
server_key_iv: "74ab92a6fa30a7d644aabb7eb8070fb6"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.874428 9343 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:05:12.875468 9358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.875682 9343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:12.875784 9343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "66966bc5789b4e2b82e13f2b10a7d454"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "1d88f6cef30fb1e429203f67d4ea2ff0"
server_key_iv: "74ab92a6fa30a7d644aabb7eb8070fb6"
server_key_version: "encryptionkey@0"
I20260619 12:05:12.875862 9343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:12.885057 9343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:12.887838 9343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:12.888005 9343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:12.888269 9343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:12.888619 9343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:12.888660 9343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.888697 9343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:12.888717 9343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:12.896341 9343 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:39185
I20260619 12:05:12.896518 9471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:39185 every 8 connection(s)
I20260619 12:05:12.896914 9343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:05:12.900624 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 9343
I20260619 12:05:12.900755 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:05:12.900939 814 external_mini_cluster.cc:1428] Setting key 37a2dce4d9259bce030a154dfec005da
Jun 19 12:05:12 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:05:12.908898 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:12.898096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:45429 (local address 127.0.203.190:33023)
0619 12:05:12.898210 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:12.898213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:12.898390 (+ 177us) server_negotiation.cc:407] Connection header received
0619 12:05:12.899831 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:12.899834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:12.899849 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:12.899941 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:12.900827 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.901601 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.903026 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.903250 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.903389 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:12.905743 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:12.905744 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:12.905745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:12.905754 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:12.907162 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.907839 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.907840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.907841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.907901 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:12.908206 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:12.908207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:12.908207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:12.908332 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:12.908383 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:12.908735 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 12:05:12.908808 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 12:05:12.909242 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:12.898013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:45429)
0619 12:05:12.898191 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:12.898237 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:12.899631 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:12.900023 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:12.900027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:12.900361 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:12.900679 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:12.900687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.901745 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.901747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:12.902829 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:12.902835 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:12.903440 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:12.903442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:12.903579 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:12.904178 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:12.904187 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:12.905635 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:12.907394 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:12.907405 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:12.907408 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:12.907748 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:12.908006 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:12.908006 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:12.908007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:12.908133 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:12.908429 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:12.908434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:12.908595 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 12:05:12.908703 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 12:05:12.908859 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260619 12:05:12.909878 9472 heartbeater.cc:347] Connected to a master server at 127.0.203.190:33023
I20260619 12:05:12.910000 9472 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:12.910249 9472 heartbeater.cc:511] Master 127.0.203.190:33023 requested a full tablet report, sending...
I20260619 12:05:12.910787 9013 ts_manager.cc:194] Registered new tserver with Master: 66966bc5789b4e2b82e13f2b10a7d454 (127.0.203.131:39185)
I20260619 12:05:12.911219 9013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:45429
I20260619 12:05:12.913195 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab.
Entry for principal oryx/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab.
WARNING: no policy specified for oryx/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab.
Entry for principal oryx/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab.
WARNING: no policy specified for oryx/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab.
Entry for principal oryx/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab.
I20260619 12:05:12.998103 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 9075
I20260619 12:05:13.002406 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 9209
I20260619 12:05:13.007455 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 9343
I20260619 12:05:13.012346 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 8981
I20260619 12:05:13.017431 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33023
--webserver_interface=127.0.203.190
--webserver_port=34027
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:13.104841 9481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:13.105094 9481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:13.105121 9481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:13.106500 9481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:13.106571 9481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:13.106587 9481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:13.106607 9481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:13.106627 9481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:13.108517 9481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:13.108825 9481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:13.109079 9481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:13.114440 9487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.114411 9486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.114452 9489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:13.114674 9481 server_base.cc:1034] running on GCE node
I20260619 12:05:13.115037 9481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:13.115339 9481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:13.116546 9481 hybrid_clock.cc:648] HybridClock initialized: now 1781870713116512 us; error 50 us; skew 500 ppm
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:13.119452 9481 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:05:13.119958 9481 webserver.cc:460] Webserver started at http://127.0.203.190:34027/ using document root <none> and password file <none>
I20260619 12:05:13.120200 9481 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:13.120265 9481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:13.121783 9481 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:13.122617 9496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.122879 9481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:13.122982 9481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "5070c32bd7f74a59a26c7ea01de30bdf"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "242ae91030034c6d73b315cfcecc406f"
server_key_iv: "7b10ceab2739212d9d814248f01dc080"
server_key_version: "encryptionkey@0"
I20260619 12:05:13.123348 9481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:13.136121 9481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:13.138553 9481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:13.138710 9481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:13.142997 9481 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:33023
I20260619 12:05:13.143036 9548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:33023 every 8 connection(s)
I20260619 12:05:13.143641 9481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:05:13.145136 9549 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:05:13.145918 9549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Bootstrap starting.
I20260619 12:05:13.153440 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 9481
I20260619 12:05:13.153883 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:40397
--local_ip_for_outbound_sockets=127.0.203.129
--tserver_master_addrs=127.0.203.190:33023
--webserver_port=38767
--webserver_interface=127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:05:13.156705 9549 log.cc:824] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:13.158948 9549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:05:13.159242 9549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Bootstrap complete.
I20260619 12:05:13.161657 9549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:13.161908 9549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5070c32bd7f74a59a26c7ea01de30bdf, State: Initialized, Role: FOLLOWER
I20260619 12:05:13.162081 9549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:13.162211 9549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:05:13.162276 9549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:05:13.162310 9549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:05:13.163554 9549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:13.163661 9549 leader_election.cc:302] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5070c32bd7f74a59a26c7ea01de30bdf; no voters:
I20260619 12:05:13.163869 9549 leader_election.cc:288] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:05:13.163928 9553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:05:13.164084 9553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [term 2 LEADER]: Becoming Leader. State: Replica: 5070c32bd7f74a59a26c7ea01de30bdf, State: Running, Role: LEADER
I20260619 12:05:13.164207 9549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:13.164222 9553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } }
I20260619 12:05:13.164777 9555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5070c32bd7f74a59a26c7ea01de30bdf. Latest consensus state: current_term: 2 leader_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } } }
I20260619 12:05:13.164923 9555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:13.165203 9554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070c32bd7f74a59a26c7ea01de30bdf" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33023 } } }
I20260619 12:05:13.165318 9554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:13.165373 9562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:13.165989 9562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:13.166188 9562 catalog_manager.cc:1259] Loaded cluster ID: 5bc70b47bc9c4de88343be3d338b6a17
I20260619 12:05:13.166389 9562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:05:13.167626 9562 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:13.167831 9562 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5070c32bd7f74a59a26c7ea01de30bdf: Loaded TSK: 0
I20260619 12:05:13.168076 9562 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:05:13.239394 9551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:13.239626 9551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:13.239660 9551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:13.240901 9551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:13.240960 9551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:13.240998 9551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:13.242487 9551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.129.keytab
--principal=oryx/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:40397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=38767
--tserver_master_addrs=127.0.203.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:13.242734 9551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:13.242957 9551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:13.248217 9578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.248294 9575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.248172 9576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:13.248816 9551 server_base.cc:1034] running on GCE node
I20260619 12:05:13.249056 9551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:13.249318 9551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:13.250487 9551 hybrid_clock.cc:648] HybridClock initialized: now 1781870713250469 us; error 50 us; skew 500 ppm
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870713, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:13.253618 9551 init.cc:377] Logged in from keytab as oryx/127.0.203.129@KRBTEST.COM (short username oryx)
I20260619 12:05:13.254122 9551 webserver.cc:460] Webserver started at http://127.0.203.129:38767/ using document root <none> and password file <none>
I20260619 12:05:13.254371 9551 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:13.254439 9551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:13.255964 9551 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:13.256608 9585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.256804 9551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:13.256882 9551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "d3f91b7742764e95884b1b6ad4db5f89"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "4a6ef91c8a4a91dfd91944dc57a35222"
server_key_iv: "da69c1c009d389dec4511381eb1e01bb"
server_key_version: "encryptionkey@0"
I20260619 12:05:13.257237 9551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:13.289870 9551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:13.292603 9551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:13.292761 9551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:13.293032 9551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:13.293375 9551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:13.293411 9551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.293434 9551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:13.293464 9551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.301371 9551 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:40397
I20260619 12:05:13.301479 9698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:40397 every 8 connection(s)
I20260619 12:05:13.301851 9551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:13.311008 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 9551
I20260619 12:05:13.311288 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:37699
--local_ip_for_outbound_sockets=127.0.203.130
--tserver_master_addrs=127.0.203.190:33023
--webserver_port=43341
--webserver_interface=127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:05:13.312183 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.302746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:41073)
0619 12:05:13.302988 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.303046 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.303661 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.304211 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.304215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.304795 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.305025 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.305030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.306408 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.306409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.307598 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.307602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.308399 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.308401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.308543 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.309145 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.309154 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.311984 (+ 2830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":117,"threads_started":1}
W20260619 12:05:13.312470 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.302850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:41073 (local address 127.0.203.190:33023)
0619 12:05:13.303098 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.303108 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.303157 (+ 49us) server_negotiation.cc:407] Connection header received
0619 12:05:13.303926 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.303934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.303988 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.304143 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.305153 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.306141 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.307725 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.308248 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.308363 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.312232 (+ 3869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:41073: BlockingRecv error: recv got EOF from 127.0.203.129:41073 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":113,"threads_started":1}
W20260619 12:05:13.312845 9699 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:33023 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.203.190:33023: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:13.318610 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.313090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:45707)
0619 12:05:13.313168 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.313184 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.313265 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.313481 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.313482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.313658 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.313746 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.313749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.314706 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.314707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.315438 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.315441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.316025 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.316026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.316097 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.316629 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.316634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.318528 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:05:13.319173 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.313138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:45707 (local address 127.0.203.190:33023)
0619 12:05:13.313181 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.313183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.313284 (+ 101us) server_negotiation.cc:407] Connection header received
0619 12:05:13.313326 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.313327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.313343 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.313414 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.313858 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.314593 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.315538 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.315891 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.316004 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.318841 (+ 2837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:45707: BlockingRecv error: recv got EOF from 127.0.203.129:45707 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:13.324873 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.319039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:36407)
0619 12:05:13.319122 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.319136 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.319194 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.319632 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.319633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.319766 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.319813 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.319815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.320761 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.320762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.321332 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.321335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.321836 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.321837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.321880 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.322412 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.322417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.324786 (+ 2369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 12:05:13.325073 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.319154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:36407 (local address 127.0.203.190:33023)
0619 12:05:13.319438 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.319441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.319454 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:05:13.319485 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.319486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.319500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.319581 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.319907 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.320650 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.321432 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.321753 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.321834 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.324970 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:36407: BlockingRecv error: recv got EOF from 127.0.203.129:36407 (error 108)
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":150,"threads_started":1}
W20260619 12:05:13.325278 9699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:05:13.397711 9703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:13.397962 9703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:13.397989 9703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:13.399917 9703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:13.400007 9703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:13.400043 9703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:13.401890 9703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.130.keytab
--principal=oryx/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:37699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=43341
--tserver_master_addrs=127.0.203.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:13.402172 9703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:13.402441 9703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:13.407701 9712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.407784 9709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.407742 9710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:13.408303 9703 server_base.cc:1034] running on GCE node
I20260619 12:05:13.408545 9703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:13.408820 9703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:13.410023 9703 hybrid_clock.cc:648] HybridClock initialized: now 1781870713409980 us; error 32 us; skew 500 ppm
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870713, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:13.413173 9703 init.cc:377] Logged in from keytab as oryx/127.0.203.130@KRBTEST.COM (short username oryx)
I20260619 12:05:13.413746 9703 webserver.cc:460] Webserver started at http://127.0.203.130:43341/ using document root <none> and password file <none>
I20260619 12:05:13.413987 9703 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:13.414062 9703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:13.415820 9703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:05:13.416653 9719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.416873 9703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:05:13.416963 9703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "fae0cf6aa41046e7ab5cca079e8c3ef2"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "9590da1952deaa61e7e60f6dc9c32602"
server_key_iv: "688ee4612145721754af6e546a156258"
server_key_version: "encryptionkey@0"
I20260619 12:05:13.417438 9703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:13.432019 9703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:13.434477 9703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:13.434638 9703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:13.434917 9703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:13.435266 9703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:13.435299 9703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.435331 9703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:13.435354 9703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.442479 9703 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:37699
I20260619 12:05:13.442670 9832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:37699 every 8 connection(s)
I20260619 12:05:13.442953 9703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:13.447651 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 9703
I20260619 12:05:13.447923 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:39185
--local_ip_for_outbound_sockets=127.0.203.131
--tserver_master_addrs=127.0.203.190:33023
--webserver_port=43873
--webserver_interface=127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:05:13.452167 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.443880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:36725 (local address 127.0.203.190:33023)
0619 12:05:13.443949 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.443952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.444252 (+ 300us) server_negotiation.cc:407] Connection header received
0619 12:05:13.444903 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.444904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.444923 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.444978 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.446102 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.446650 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.447822 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.448052 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.448132 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.452107 (+ 3975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:36725: BlockingRecv error: recv got EOF from 127.0.203.130:36725 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:05:13.452260 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.443784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:36725)
0619 12:05:13.444083 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.444130 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.444786 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.445069 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.445074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.445722 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.445996 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.446001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.446740 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.446741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.447671 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.447673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.448207 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.448208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.448339 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.448947 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.448961 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.451977 (+ 3016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":149,"threads_started":1}
W20260619 12:05:13.452582 9833 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:33023 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.203.190:33023: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:13.457906 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.452845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:46711)
0619 12:05:13.452902 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.452914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.452955 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.453174 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.453175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.453334 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.453402 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.453405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.454407 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.454408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.454971 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.454973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.455483 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.455484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.455540 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.456133 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.456139 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.457848 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 12:05:13.458078 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.452925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:46711 (local address 127.0.203.190:33023)
0619 12:05:13.452982 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.452984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.452995 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:13.453032 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.453033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.453045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.453128 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.453504 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.454297 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.455062 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.455391 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.455466 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.457991 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:46711: BlockingRecv error: recv got EOF from 127.0.203.130:46711 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:13.462520 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.458162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:56207)
0619 12:05:13.458357 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.458367 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.458433 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.458563 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.458564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.458689 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.458720 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.458721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.459352 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.459353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.459868 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.459869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.460221 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.460222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.460279 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.460699 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.460703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.462449 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":140,"threads_started":1}
W20260619 12:05:13.462600 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.458303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:56207 (local address 127.0.203.190:33023)
0619 12:05:13.458356 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.458358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.458408 (+ 50us) server_negotiation.cc:407] Connection header received
0619 12:05:13.458499 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.458500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.458510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.458556 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.458783 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.459278 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.459938 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.460166 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.460223 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.462541 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:56207: BlockingRecv error: recv got EOF from 127.0.203.130:56207 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 12:05:13.462667 9833 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:05:13.541117 9836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:13.541421 9836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:13.541455 9836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:13.542976 9836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:13.543061 9836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:13.543097 9836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:13.545001 9836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/krb5kdc/oryx_127.0.203.131.keytab
--principal=oryx/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=43873
--tserver_master_addrs=127.0.203.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:13.545324 9836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:13.545615 9836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:13.551622 9843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.551910 9845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:13.552091 9842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:13.553078 9836 server_base.cc:1034] running on GCE node
I20260619 12:05:13.553431 9836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:13.553802 9836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:13.554967 9836 hybrid_clock.cc:648] HybridClock initialized: now 1781870713554957 us; error 34 us; skew 500 ppm
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870713, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:13.557925 9836 init.cc:377] Logged in from keytab as oryx/127.0.203.131@KRBTEST.COM (short username oryx)
I20260619 12:05:13.558457 9836 webserver.cc:460] Webserver started at http://127.0.203.131:43873/ using document root <none> and password file <none>
I20260619 12:05:13.558715 9836 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:13.558787 9836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:13.560478 9836 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:13.561223 9852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.561435 9836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:13.561527 9836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "66966bc5789b4e2b82e13f2b10a7d454"
format_stamp: "Formatted at 2026-06-19 12:05:12 on dist-test-slave-44vj"
server_key: "1d88f6cef30fb1e429203f67d4ea2ff0"
server_key_iv: "74ab92a6fa30a7d644aabb7eb8070fb6"
server_key_version: "encryptionkey@0"
I20260619 12:05:13.561956 9836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:13.577868 9836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:13.580999 9836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:13.581226 9836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:13.581532 9836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:13.581962 9836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:13.582000 9836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.582043 9836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:13.582068 9836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:13.589546 9836 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:39185
I20260619 12:05:13.589636 9965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:39185 every 8 connection(s)
I20260619 12:05:13.590062 9836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:05:13.594518 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 9836
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:05:13.598794 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.591020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:58923 (local address 127.0.203.190:33023)
0619 12:05:13.591110 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.591112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.591326 (+ 214us) server_negotiation.cc:407] Connection header received
0619 12:05:13.591904 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.591905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.591916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.591961 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.593000 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.593495 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.594680 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.594987 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.595080 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.598720 (+ 3640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:58923: BlockingRecv error: recv got EOF from 127.0.203.131:58923 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:05:13.598927 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.590992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:58923)
0619 12:05:13.591176 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.591220 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.591794 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.592036 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.592040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.592670 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.592898 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.592903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.593586 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.593587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.594520 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.594523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.595125 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.595127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.595331 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.595947 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.595959 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.598637 (+ 2678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
W20260619 12:05:13.599226 9966 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:33023 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.203.190:33023: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
I20260619 12:05:13.603348 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.595242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52346 (local address 127.0.203.190:33023)
0619 12:05:13.595310 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.595311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.595442 (+ 131us) server_negotiation.cc:407] Connection header received
0619 12:05:13.595540 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.595541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.595551 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.595628 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.596326 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.596888 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.597794 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.598078 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.598160 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.599577 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:13.599584 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:13.599584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:13.599601 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:13.601516 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:13.601999 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:13.602003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:13.602004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:13.602170 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:13.602379 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:13.602380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:13.602380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:13.602713 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:13.602842 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:13.603014 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 12:05:13.603261 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:13.605430 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.599491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:34035)
0619 12:05:13.599893 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.599904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.599952 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.600143 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.600146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.600289 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.600351 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.600354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.601344 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.601346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.602173 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.602178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.602684 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.602685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.602765 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.603288 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.603292 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.605370 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384}
W20260619 12:05:13.605558 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.599572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:34035 (local address 127.0.203.190:33023)
0619 12:05:13.599626 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.599628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.599953 (+ 325us) server_negotiation.cc:407] Connection header received
0619 12:05:13.600008 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.600008 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.600020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.600078 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.600476 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.601203 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.602267 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.602564 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.602645 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.605488 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:34035: BlockingRecv error: recv got EOF from 127.0.203.131:34035 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:13 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:13.610522 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:13.605880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:48433)
0619 12:05:13.605936 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:13.605949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:13.605992 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:13.606155 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:13.606155 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:13.606260 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:13.606299 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.606300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.607179 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.607179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.607848 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:13.607850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.608269 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.608271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:13.608325 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.608754 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:13.608758 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:13.610465 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 12:05:13.610649 9702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:13.605998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:48433 (local address 127.0.203.190:33023)
0619 12:05:13.606048 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:13.606050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:13.606059 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:05:13.606081 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:13.606082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:13.606092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:13.606141 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:13.606369 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.607103 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.607953 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:13.608156 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:13.608228 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:13.610577 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:48433: BlockingRecv error: recv got EOF from 127.0.203.131:48433 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 12:05:13.610715 9966 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 12:05:14 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:14 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:14.331576 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:14.325892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:34759)
0619 12:05:14.326128 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:14.326140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:14.326217 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:14.326400 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:14.326400 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:14.326632 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:14.326702 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:14.326705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.327514 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.327515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:14.328170 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:14.328173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.328633 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.328634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:14.328697 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:14.329310 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:14.329315 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:14.331494 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":174,"threads_started":1}
W20260619 12:05:14.331701 9971 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:14.326090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:34759 (local address 127.0.203.190:33023)
0619 12:05:14.326222 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:14.326224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:14.326238 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:05:14.326277 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:14.326278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:14.326289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:14.326382 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:14.326817 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.327417 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.328295 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.328523 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.328598 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:14.331619 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:34759: BlockingRecv error: recv got EOF from 127.0.203.129:34759 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
Jun 19 12:05:14 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:14 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:14.469836 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:14.463293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:42047)
0619 12:05:14.463518 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:14.463530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:14.463613 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:14.463790 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:14.463791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:14.463994 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:14.464062 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:14.464064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.465205 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.465207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:14.465975 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:14.465980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.466481 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.466482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:14.466549 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:14.467133 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:14.467139 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:14.469745 (+ 2606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":126,"threads_started":1}
W20260619 12:05:14.470031 9971 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:14.463443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:42047 (local address 127.0.203.190:33023)
0619 12:05:14.463501 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:14.463503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:14.463610 (+ 107us) server_negotiation.cc:407] Connection header received
0619 12:05:14.463671 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:14.463672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:14.463686 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:14.463771 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:14.464226 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.465043 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.466098 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.466381 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.466460 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:14.469947 (+ 3487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:42047: BlockingRecv error: recv got EOF from 127.0.203.130:42047 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Jun 19 12:05:14 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:14 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:14.616463 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:14.611188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:38265)
0619 12:05:14.611346 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:14.611384 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:14.611456 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:14.611610 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:14.611611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:14.611806 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:14.611875 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:14.611877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.612602 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.612603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:14.613223 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:14.613225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.613650 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.613652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:14.613712 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:14.614250 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:14.614256 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:14.616379 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
W20260619 12:05:14.616583 9971 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:14.611288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:38265 (local address 127.0.203.190:33023)
0619 12:05:14.611348 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:14.611350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:14.611458 (+ 108us) server_negotiation.cc:407] Connection header received
0619 12:05:14.611532 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:14.611533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:14.611546 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:14.611595 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:14.611985 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.612508 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.613341 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:14.613540 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:14.613615 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:14.616509 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:38265: BlockingRecv error: recv got EOF from 127.0.203.131:38265 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 12:05:15 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:15 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:15.337682 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:15.332169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:39903)
0619 12:05:15.332326 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:15.332337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:15.332401 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:15.332630 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:15.332631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:15.332828 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:15.332896 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:15.332898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.333627 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.333628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:15.334237 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:15.334239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.334659 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.334660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:15.334702 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:15.335259 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:15.335264 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:15.337599 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
W20260619 12:05:15.337893 9975 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:15.332280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:39903 (local address 127.0.203.190:33023)
0619 12:05:15.332476 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:15.332478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:15.332488 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:15.332512 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:15.332512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:15.332523 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:15.332615 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:15.332994 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.333531 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.334361 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.334581 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.334656 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:15.337796 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:39903: BlockingRecv error: recv got EOF from 127.0.203.129:39903 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
Jun 19 12:05:15 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:15 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:15.476389 9976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:15.470457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:47101)
0619 12:05:15.470616 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:15.470628 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:15.470696 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:15.470866 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:15.470867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:15.471101 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:15.471170 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:15.471173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.472058 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.472059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:15.472663 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:15.472666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.473256 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.473257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:15.473318 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:15.473859 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:15.473869 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:15.476219 (+ 2350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
W20260619 12:05:15.476475 9975 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:15.470628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:47101 (local address 127.0.203.190:33023)
0619 12:05:15.470693 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:15.470695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:15.470752 (+ 57us) server_negotiation.cc:407] Connection header received
0619 12:05:15.470781 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:15.470782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:15.470793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:15.470846 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:15.471281 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.471922 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.472850 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.473127 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.473216 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:15.476393 (+ 3177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:47101: BlockingRecv error: recv got EOF from 127.0.203.130:47101 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 12:05:15 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:15 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:15.622488 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:15.617114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:42713)
0619 12:05:15.617329 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:15.617346 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:15.617437 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:15.617636 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:15.617638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:15.617829 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:15.617897 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:15.617899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.618641 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.618642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:15.619225 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:15.619226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.619720 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.619722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:15.619781 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:15.620283 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:15.620288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:15.622427 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
W20260619 12:05:15.622608 9975 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:15.617311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:42713 (local address 127.0.203.190:33023)
0619 12:05:15.617391 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:15.617393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:15.617417 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:05:15.617501 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:15.617502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:15.617516 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:15.617578 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:15.618029 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.618558 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.619337 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:15.619616 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:15.619694 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:15.622532 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:42713: BlockingRecv error: recv got EOF from 127.0.203.131:42713 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 12:05:16 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:16 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:16.343788 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:16.338313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:48257)
0619 12:05:16.338475 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:16.338486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:16.338558 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:16.338716 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:16.338717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:16.338906 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:16.338974 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:16.338976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.339786 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.339787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:16.340406 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:16.340409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.340860 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.340862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:16.340934 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:16.341466 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:16.341471 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:16.343692 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
W20260619 12:05:16.343915 9979 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:16.338366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:48257 (local address 127.0.203.190:33023)
0619 12:05:16.338511 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:16.338513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:16.338549 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:05:16.338622 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:16.338622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:16.338634 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:16.338691 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:16.339042 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.339680 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.340543 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.340758 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.340835 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:16.343829 (+ 2994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:48257: BlockingRecv error: recv got EOF from 127.0.203.129:48257 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
Jun 19 12:05:16 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:16 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:16.482745 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:16.477215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:46055)
0619 12:05:16.477375 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:16.477387 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:16.477455 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:16.477631 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:16.477631 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:16.477839 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:16.477912 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:16.477914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.478963 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.478964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:16.479612 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:16.479615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.480000 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.480000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:16.480044 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:16.480574 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:16.480578 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:16.482657 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
W20260619 12:05:16.482897 9979 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:16.477373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:46055 (local address 127.0.203.190:33023)
0619 12:05:16.477434 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:16.477436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:16.477451 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:05:16.477517 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:16.477518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:16.477529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:16.477612 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:16.478056 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.478879 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.479716 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.479923 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.479996 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:16.482821 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:46055: BlockingRecv error: recv got EOF from 127.0.203.130:46055 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 12:05:16 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:16 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:16.628320 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:16.623093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:57021)
0619 12:05:16.623298 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:16.623316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:16.623480 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:16.623790 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:16.623791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:16.624041 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:16.624121 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:16.624124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.624776 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.624776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:16.625387 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:16.625390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.625720 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.625721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:16.625766 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:16.626237 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:16.626241 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:16.628234 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
W20260619 12:05:16.628418 9979 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:16.623247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:57021 (local address 127.0.203.190:33023)
0619 12:05:16.623346 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:16.623350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:16.623463 (+ 113us) server_negotiation.cc:407] Connection header received
0619 12:05:16.623678 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:16.623680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:16.623697 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:16.623769 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:16.624215 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.624703 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.625466 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:16.625656 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:16.625713 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:16.628347 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:57021: BlockingRecv error: recv got EOF from 127.0.203.131:57021 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 19 12:05:17 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:17 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:17.353545 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:17.344479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:48481)
0619 12:05:17.344655 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:17.344667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:17.344738 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:17.344962 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:17.344965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:17.345211 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:17.345282 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:17.345284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.345978 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.345979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:17.346587 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:17.346589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.346930 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.346930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:17.346976 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:17.351268 (+ 4292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:17.351275 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:17.353471 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
W20260619 12:05:17.353744 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:17.344619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:48481 (local address 127.0.203.190:33023)
0619 12:05:17.344761 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:17.344763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:17.344774 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:17.344802 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:17.344803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:17.344813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:17.344904 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:17.345366 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.345874 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.346666 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.346861 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.346916 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:17.353651 (+ 6735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:48481: BlockingRecv error: recv got EOF from 127.0.203.129:48481 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
Jun 19 12:05:17 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:17 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:17.488936 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:17.483511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:32813)
0619 12:05:17.483732 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:17.483749 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:17.483874 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:17.484052 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:17.484053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:17.484313 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:17.484396 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:17.484400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.485074 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.485075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:17.485661 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:17.485663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.486099 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.486100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:17.486143 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:17.486687 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:17.486692 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:17.488841 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":79,"threads_started":1}
W20260619 12:05:17.489005 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:17.483643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:32813 (local address 127.0.203.190:33023)
0619 12:05:17.483731 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:17.483733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:17.483862 (+ 129us) server_negotiation.cc:407] Connection header received
0619 12:05:17.483937 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:17.483939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:17.483952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:17.484028 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:17.484482 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.485000 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.485815 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.486032 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.486093 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:17.488953 (+ 2860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:32813: BlockingRecv error: recv got EOF from 127.0.203.130:32813 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 12:05:17 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:17 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:17.635568 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:17.628889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:57351)
0619 12:05:17.629173 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:17.629190 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:17.629320 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:17.629509 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:17.629510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:17.629767 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:17.629854 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:17.629856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.630553 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.630554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:17.631534 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:17.631537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.632085 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.632086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:17.632156 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:17.632774 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:17.632780 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:17.635324 (+ 2544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":76,"threads_started":1}
W20260619 12:05:17.635746 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:17.629057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:57351 (local address 127.0.203.190:33023)
0619 12:05:17.629160 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:17.629162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:17.629290 (+ 128us) server_negotiation.cc:407] Connection header received
0619 12:05:17.629394 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:17.629395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:17.629411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:17.629478 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:17.629944 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.630448 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.631658 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:17.631981 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:17.632077 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:17.635670 (+ 3593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:57351: BlockingRecv error: recv got EOF from 127.0.203.131:57351 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 19 12:05:18 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:18 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:18.359314 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:18.354101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:47633)
0619 12:05:18.354276 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:18.354295 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:18.354400 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:18.354583 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:18.354583 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:18.354767 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:18.354832 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:18.354834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.355504 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.355505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:18.356106 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:18.356107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.356525 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.356526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:18.356573 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:18.357090 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:18.357095 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:18.359229 (+ 2134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
W20260619 12:05:18.359526 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:18.354175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:47633 (local address 127.0.203.190:33023)
0619 12:05:18.354282 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:18.354290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:18.354377 (+ 87us) server_negotiation.cc:407] Connection header received
0619 12:05:18.354482 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:18.354483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:18.354496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:18.354570 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:18.354896 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.355433 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.356227 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.356442 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.356516 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:18.359443 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:47633: BlockingRecv error: recv got EOF from 127.0.203.129:47633 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":49,"threads_started":1}
Jun 19 12:05:18 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:18 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:18.495343 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:18.489643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:34313)
0619 12:05:18.489891 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:18.489909 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:18.490007 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:18.490217 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:18.490219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:18.490421 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:18.490477 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:18.490479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.491238 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.491240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:18.491860 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:18.491862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.492303 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.492304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:18.492374 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:18.493033 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:18.493038 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:18.495247 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":107,"threads_started":1}
W20260619 12:05:18.495442 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:18.489890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:34313 (local address 127.0.203.190:33023)
0619 12:05:18.489975 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:18.489978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:18.489989 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:18.490076 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:18.490077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:18.490089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:18.490151 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:18.490568 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.491134 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.491964 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.492186 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.492248 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:18.495354 (+ 3106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:34313: BlockingRecv error: recv got EOF from 127.0.203.130:34313 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 12:05:18 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:18 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:18.641650 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:18.636269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:49263)
0619 12:05:18.636472 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:18.636498 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:18.636602 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:18.636765 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:18.636766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:18.636977 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:18.637039 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:18.637041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.637747 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.637748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:18.638442 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:18.638446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.638815 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.638816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:18.638871 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:18.639482 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:18.639486 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:18.641580 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
W20260619 12:05:18.641779 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:18.636415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:49263 (local address 127.0.203.190:33023)
0619 12:05:18.636488 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:18.636491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:18.636582 (+ 91us) server_negotiation.cc:407] Connection header received
0619 12:05:18.636683 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:18.636684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:18.636701 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:18.636750 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:18.637115 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.637661 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.638551 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:18.638754 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:18.638811 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:18.641700 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:49263: BlockingRecv error: recv got EOF from 127.0.203.131:49263 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 12:05:19 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:19 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:19.365566 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:19.360014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:56139)
0619 12:05:19.360162 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:19.360175 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:19.360272 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:19.360610 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:19.360612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:19.360817 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:19.360889 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:19.360892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.361854 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.361856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:19.362429 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:19.362431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.362863 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.362864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:19.362923 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:19.363537 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:19.363544 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:19.365501 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
W20260619 12:05:19.365916 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:19.360222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:56139 (local address 127.0.203.190:33023)
0619 12:05:19.360393 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:19.360395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:19.360407 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:05:19.360444 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:19.360444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:19.360457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:19.360556 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:19.361055 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.361782 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.362529 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.362757 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.362825 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:19.365787 (+ 2962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:56139: BlockingRecv error: recv got EOF from 127.0.203.129:56139 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
Jun 19 12:05:19 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:19 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:19.501194 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:19.496105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:45673)
0619 12:05:19.496398 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:19.496410 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:19.496487 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:19.496676 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:19.496677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:19.496883 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:19.496971 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:19.496973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.497688 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.497689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:19.498239 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:19.498240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.498636 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.498637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:19.498717 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:19.499236 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:19.499242 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:19.501126 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":76,"threads_started":1}
W20260619 12:05:19.501345 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:19.496227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:45673 (local address 127.0.203.190:33023)
0619 12:05:19.496408 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:19.496410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:19.496464 (+ 54us) server_negotiation.cc:407] Connection header received
0619 12:05:19.496567 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:19.496568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:19.496583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:19.496650 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:19.497042 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.497599 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.498323 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.498519 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.498577 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:19.501253 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:45673: BlockingRecv error: recv got EOF from 127.0.203.130:45673 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 12:05:19 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:19 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:19.646687 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:19.642110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:38023)
0619 12:05:19.642253 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:19.642284 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:19.642343 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:19.642482 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:19.642482 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:19.642649 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:19.642702 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:19.642705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.643451 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.643452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:19.644038 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:19.644040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.644440 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.644441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:19.644495 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:19.644975 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:19.644979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:19.646626 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
W20260619 12:05:19.646842 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:19.642227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:38023 (local address 127.0.203.190:33023)
0619 12:05:19.642294 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:19.642296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:19.642331 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:05:19.642395 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:19.642396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:19.642410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:19.642464 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:19.642767 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.643310 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.644123 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:19.644325 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:19.644385 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:19.646758 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:38023: BlockingRecv error: recv got EOF from 127.0.203.131:38023 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 12:05:20 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:20 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:20.371138 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:20.366107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:58855)
0619 12:05:20.366256 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:20.366268 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:20.366334 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:20.366552 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:20.366552 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:20.366711 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:20.366765 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:20.366767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.367674 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.367675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:20.368245 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:20.368247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.368630 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.368631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:20.368672 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:20.369205 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:20.369211 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:20.371070 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
W20260619 12:05:20.371425 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:20.366212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:58855 (local address 127.0.203.190:33023)
0619 12:05:20.366371 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:20.366373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:20.366385 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:05:20.366417 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:20.366418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:20.366433 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:20.366537 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:20.366853 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.367585 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.368346 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.368564 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.368624 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:20.371279 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:58855: BlockingRecv error: recv got EOF from 127.0.203.129:58855 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
Jun 19 12:05:20 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:20 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:20.506347 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:20.501772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:37843)
0619 12:05:20.501928 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:20.501941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:20.502016 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:20.502157 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:20.502158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:20.502323 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:20.502368 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:20.502369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.503039 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.503040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:20.503587 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:20.503588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.503914 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.503915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:20.503971 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:20.504481 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:20.504485 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:20.506263 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
W20260619 12:05:20.506426 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:20.501890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:37843 (local address 127.0.203.190:33023)
0619 12:05:20.501986 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:20.501989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:20.501999 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:20.502081 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:20.502082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:20.502096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:20.502142 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:20.502443 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.502964 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.503664 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.503856 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.503911 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:20.506370 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:37843: BlockingRecv error: recv got EOF from 127.0.203.130:37843 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 12:05:20 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:20 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:20.652617 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:20.647451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:60551)
0619 12:05:20.647624 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:20.647641 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:20.647762 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:20.647909 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:20.647910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:20.648108 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:20.648159 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:20.648161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.648821 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.648822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:20.649344 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:20.649346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.649671 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.649673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:20.649723 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:20.650338 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:20.650342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:20.652534 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
W20260619 12:05:20.652704 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:20.647597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:60551 (local address 127.0.203.190:33023)
0619 12:05:20.647703 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:20.647706 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:20.647735 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:05:20.647832 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:20.647833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:20.647847 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:20.647895 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:20.648238 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.648736 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.649419 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:20.649610 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:20.649665 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:20.652651 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:60551: BlockingRecv error: recv got EOF from 127.0.203.131:60551 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 12:05:21 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:21 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:21.377497 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:21.371771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:58211)
0619 12:05:21.371987 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:21.372001 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:21.372090 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:21.372306 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:21.372308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:21.372476 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:21.372530 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:21.372531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.373272 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.373273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:21.373844 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:21.373846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.374416 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.374418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:21.374496 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:21.375070 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:21.375076 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:21.377422 (+ 2346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
W20260619 12:05:21.377857 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:21.371945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:58211 (local address 127.0.203.190:33023)
0619 12:05:21.372078 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:21.372080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:21.372091 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:21.372169 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:21.372170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:21.372182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:21.372245 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:21.372609 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.373172 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.374011 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.374308 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.374384 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:21.377753 (+ 3369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:58211: BlockingRecv error: recv got EOF from 127.0.203.129:58211 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
Jun 19 12:05:21 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:21 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:21.512311 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:21.506959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:58787)
0619 12:05:21.507135 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:21.507150 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:21.507253 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:21.507470 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:21.507471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:21.507644 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:21.507689 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:21.507691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.508520 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.508521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:21.509046 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:21.509048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.509378 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.509379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:21.509440 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:21.510027 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:21.510031 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:21.512241 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
W20260619 12:05:21.512488 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:21.507076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:58787 (local address 127.0.203.190:33023)
0619 12:05:21.507192 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:21.507194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:21.507226 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:05:21.507324 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:21.507325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:21.507338 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:21.507456 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:21.507768 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.508443 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.509127 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.509317 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.509373 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:21.512406 (+ 3033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:58787: BlockingRecv error: recv got EOF from 127.0.203.130:58787 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Jun 19 12:05:21 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:21 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:21.658346 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:21.653302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:39981)
0619 12:05:21.653562 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:21.653579 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:21.653691 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:21.653848 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:21.653849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:21.654028 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:21.654073 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:21.654075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.654744 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.654745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:21.655269 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:21.655271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.655638 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.655639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:21.655691 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:21.656223 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:21.656228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:21.658269 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":167,"threads_started":1}
W20260619 12:05:21.658533 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:21.653475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:39981 (local address 127.0.203.190:33023)
0619 12:05:21.653600 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:21.653603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:21.653657 (+ 54us) server_negotiation.cc:407] Connection header received
0619 12:05:21.653769 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:21.653770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:21.653785 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:21.653833 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:21.654150 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.654665 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.655351 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:21.655576 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:21.655633 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:21.658452 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:39981: BlockingRecv error: recv got EOF from 127.0.203.131:39981 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 19 12:05:22 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:22 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:22.389431 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:22.378085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:37053)
0619 12:05:22.378226 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:22.378236 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:22.378290 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:22.378547 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:22.378552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:22.378823 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:22.378879 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:22.378882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.379806 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.379807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:22.380389 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:22.380390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.380850 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.380852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:22.380926 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:22.386623 (+ 5697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:22.386630 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:22.389310 (+ 2680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
W20260619 12:05:22.389545 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:22.378184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:37053 (local address 127.0.203.190:33023)
0619 12:05:22.378348 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:22.378350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:22.378360 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:05:22.378384 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:22.378385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:22.378398 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:22.378488 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:22.379013 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.379682 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.380500 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.380728 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.380793 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:22.389468 (+ 8675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:37053: BlockingRecv error: recv got EOF from 127.0.203.129:37053 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
Jun 19 12:05:22 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:22 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:22.518199 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:22.512894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:45197)
0619 12:05:22.513076 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:22.513092 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:22.513225 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:22.513371 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:22.513372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:22.513582 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:22.513646 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:22.513650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.514360 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.514363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:22.514909 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:22.514912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.515395 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.515397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:22.515489 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:22.516012 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:22.516016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:22.518120 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
W20260619 12:05:22.518383 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:22.512976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:45197 (local address 127.0.203.190:33023)
0619 12:05:22.513082 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:22.513085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:22.513210 (+ 125us) server_negotiation.cc:407] Connection header received
0619 12:05:22.513299 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:22.513300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:22.513315 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:22.513360 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:22.513724 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.514279 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.514987 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.515210 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.515285 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:22.518280 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:45197: BlockingRecv error: recv got EOF from 127.0.203.130:45197 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 19 12:05:22 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:22 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:22.664011 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:22.658819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:48223)
0619 12:05:22.658977 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:22.658989 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:22.659057 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:22.659247 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:22.659272 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:22.659488 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:22.659542 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:22.659544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.660318 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.660319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:22.660923 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:22.660926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.661319 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.661320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:22.661371 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:22.661909 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:22.661914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:22.663920 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
W20260619 12:05:22.664121 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:22.658881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:48223 (local address 127.0.203.190:33023)
0619 12:05:22.658941 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:22.658943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:22.659054 (+ 111us) server_negotiation.cc:407] Connection header received
0619 12:05:22.659118 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:22.659119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:22.659141 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:22.659205 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:22.659625 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.660218 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.661017 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:22.661248 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:22.661309 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:22.664045 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:48223: BlockingRecv error: recv got EOF from 127.0.203.131:48223 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 12:05:23 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:23 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:23.395195 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:23.390193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:59503)
0619 12:05:23.390352 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:23.390364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:23.390431 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:23.390641 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:23.390642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:23.390838 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:23.390907 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:23.390909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.391699 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.391701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:23.392296 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:23.392298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.392704 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.392705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:23.392760 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:23.393303 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:23.393308 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:23.395108 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
W20260619 12:05:23.395453 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:23.390267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:59503 (local address 127.0.203.190:33023)
0619 12:05:23.390430 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:23.390432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:23.390451 (+ 19us) server_negotiation.cc:407] Connection header received
0619 12:05:23.390487 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:23.390487 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:23.390499 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:23.390622 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:23.390976 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.391565 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.392395 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.392629 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.392692 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:23.395324 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:59503: BlockingRecv error: recv got EOF from 127.0.203.129:59503 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
Jun 19 12:05:23 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:23 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:23.524367 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:23.518889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:50443)
0619 12:05:23.519059 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:23.519071 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:23.519141 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:23.519467 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:23.519469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:23.519707 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:23.519779 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:23.519782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.520586 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.520587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:23.521149 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:23.521152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.521553 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.521554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:23.521606 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:23.522143 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:23.522147 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:23.524292 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
W20260619 12:05:23.524560 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:23.519068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:50443 (local address 127.0.203.190:33023)
0619 12:05:23.519184 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:23.519186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:23.519202 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:05:23.519236 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:23.519237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:23.519250 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:23.519346 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:23.519885 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.520485 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.521251 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.521476 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.521546 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:23.524457 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:50443: BlockingRecv error: recv got EOF from 127.0.203.130:50443 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 19 12:05:23 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:23 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:23.669960 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:23.664576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:59453)
0619 12:05:23.664775 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:23.664789 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:23.664864 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:23.665031 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:23.665033 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:23.665275 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:23.665426 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:23.665429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.666234 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.666235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:23.666824 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:23.666826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.667190 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.667191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:23.667236 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:23.667729 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:23.667733 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:23.669885 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
W20260619 12:05:23.670131 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:23.664757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:59453 (local address 127.0.203.190:33023)
0619 12:05:23.664858 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:23.664861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:23.664877 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:05:23.664947 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:23.664948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:23.664962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:23.665012 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:23.665546 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.666158 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.666916 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:23.667119 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:23.667184 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:23.670050 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:59453: BlockingRecv error: recv got EOF from 127.0.203.131:59453 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 12:05:24 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:24 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:24.401362 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:24.395736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:46051)
0619 12:05:24.395910 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:24.395925 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:24.395992 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:24.396203 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:24.396204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:24.396503 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:24.396572 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:24.396576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.397425 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.397426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:24.398029 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:24.398031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.398468 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.398468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:24.398524 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:24.399058 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:24.399062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:24.401272 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
W20260619 12:05:24.401479 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:24.395860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:46051 (local address 127.0.203.190:33023)
0619 12:05:24.396036 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:24.396038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:24.396049 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:24.396086 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:24.396087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:24.396099 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:24.396181 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:24.396698 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.397313 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.398135 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.398368 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.398437 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:24.401401 (+ 2964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:46051: BlockingRecv error: recv got EOF from 127.0.203.129:46051 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
Jun 19 12:05:24 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:24 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:24.529862 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:24.524988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:46729)
0619 12:05:24.525127 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:24.525138 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:24.525203 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:24.525395 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:24.525396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:24.525586 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:24.525643 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:24.525646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.526367 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.526369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:24.526905 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:24.526907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.527249 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.527250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:24.527301 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:24.527811 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:24.527816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:24.529760 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
W20260619 12:05:24.530026 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:24.525101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:46729 (local address 127.0.203.190:33023)
0619 12:05:24.525215 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:24.525217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:24.525232 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:05:24.525272 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:24.525273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:24.525287 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:24.525377 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:24.525767 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.526283 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.526991 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.527185 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.527243 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:24.529933 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:46729: BlockingRecv error: recv got EOF from 127.0.203.130:46729 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 19 12:05:24 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:24 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:24.676421 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:24.670685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:38677)
0619 12:05:24.670856 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:24.670872 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:24.670994 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:24.671155 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:24.671156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:24.671420 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:24.671483 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:24.671485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.672215 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.672216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:24.672873 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:24.672876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.673335 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.673337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:24.673430 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:24.673991 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:24.673997 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:24.676316 (+ 2319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
W20260619 12:05:24.676501 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:24.670814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:38677 (local address 127.0.203.190:33023)
0619 12:05:24.670911 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:24.670913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:24.670974 (+ 61us) server_negotiation.cc:407] Connection header received
0619 12:05:24.671066 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:24.671067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:24.671089 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:24.671141 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:24.671555 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.672131 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.673000 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:24.673246 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:24.673301 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:24.676434 (+ 3133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:38677: BlockingRecv error: recv got EOF from 127.0.203.131:38677 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 12:05:25 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:25 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:25.407493 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:25.402049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:57087)
0619 12:05:25.402209 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:25.402220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:25.402275 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:25.402522 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:25.402523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:25.402753 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:25.402820 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:25.402823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.403749 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.403750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:25.404324 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:25.404326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.404713 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.404714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:25.404758 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:25.405317 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:25.405321 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:25.407347 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
W20260619 12:05:25.407636 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:25.402184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:57087 (local address 127.0.203.190:33023)
0619 12:05:25.402330 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:25.402332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:25.402343 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:25.402375 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:25.402376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:25.402389 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:25.402494 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:25.402909 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.403653 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.404420 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.404640 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.404708 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:25.407543 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:57087: BlockingRecv error: recv got EOF from 127.0.203.129:57087 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
Jun 19 12:05:25 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:25 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:25.535413 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:25.530703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:57557)
0619 12:05:25.530837 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:25.530848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:25.530909 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:25.531057 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:25.531057 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:25.531227 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:25.531269 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:25.531271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.531962 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.531963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:25.532473 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:25.532475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.532792 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.532793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:25.532844 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:25.533382 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:25.533387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:25.535284 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
W20260619 12:05:25.535478 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:25.530840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:57557 (local address 127.0.203.190:33023)
0619 12:05:25.530926 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:25.530928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:25.530950 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:05:25.530984 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:25.530984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:25.530996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:25.531039 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:25.531336 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.531889 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.532543 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.532732 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.532787 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:25.535424 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:57557: BlockingRecv error: recv got EOF from 127.0.203.130:57557 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 12:05:25 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:25 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:25.681998 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:25.677036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:33761)
0619 12:05:25.677223 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:25.677237 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:25.677315 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:25.677461 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:25.677462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:25.677671 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:25.677728 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:25.677730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.678404 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.678405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:25.678946 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:25.678948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.679265 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.679266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:25.679321 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:25.679838 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:25.679842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:25.681933 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
W20260619 12:05:25.682116 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:25.677203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:33761 (local address 127.0.203.190:33023)
0619 12:05:25.677295 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:25.677298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:25.677311 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:05:25.677378 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:25.677380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:25.677397 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:25.677447 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:25.677802 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.678334 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.679019 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:25.679208 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:25.679262 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:25.682029 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:33761: BlockingRecv error: recv got EOF from 127.0.203.131:33761 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 12:05:26 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:26 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:26.413786 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:26.408314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:33297)
0619 12:05:26.408497 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:26.408533 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:26.408635 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:26.408851 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:26.408852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:26.409029 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:26.409084 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:26.409086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.409856 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.409858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:26.410668 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:26.410672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.411129 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.411130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:26.411180 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:26.411710 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:26.411714 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:26.413682 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
W20260619 12:05:26.413954 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:26.408445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:33297 (local address 127.0.203.190:33023)
0619 12:05:26.408617 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:26.408619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:26.408633 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:05:26.408702 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:26.408703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:26.408719 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:26.408793 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:26.409174 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.409755 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.410794 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.411062 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.411127 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:26.413827 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:33297: BlockingRecv error: recv got EOF from 127.0.203.129:33297 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
Jun 19 12:05:26 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:26 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:26.540937 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:26.535966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:46801)
0619 12:05:26.536111 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:26.536120 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:26.536171 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:26.536325 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:26.536326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:26.536493 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:26.536541 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:26.536543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.537258 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.537258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:26.537844 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:26.537846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.538259 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.538261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:26.538319 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:26.538830 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:26.538834 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:26.540842 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
W20260619 12:05:26.541010 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:26.536130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:46801 (local address 127.0.203.190:33023)
0619 12:05:26.536197 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:26.536199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:26.536208 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:05:26.536237 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:26.536238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:26.536250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:26.536313 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:26.536647 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.537186 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.537948 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.538151 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.538258 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:26.540950 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:46801: BlockingRecv error: recv got EOF from 127.0.203.130:46801 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 12:05:26 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:26 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:26.687963 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:26.682602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:42271)
0619 12:05:26.682769 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:26.682785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:26.682886 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:26.683032 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:26.683039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:26.683243 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:26.683307 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:26.683309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.684047 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.684048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:26.684657 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:26.684658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.685133 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.685134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:26.685191 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:26.685758 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:26.685762 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:26.687873 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
W20260619 12:05:26.688066 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:26.682720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:42271 (local address 127.0.203.190:33023)
0619 12:05:26.682794 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:26.682796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:26.682867 (+ 71us) server_negotiation.cc:407] Connection header received
0619 12:05:26.682940 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:26.682941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:26.682955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:26.683012 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:26.683418 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.683946 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.684766 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:26.685020 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:26.685093 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:26.687994 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:42271: BlockingRecv error: recv got EOF from 127.0.203.131:42271 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 12:05:27 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:27 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:27.423187 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:27.414716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:45237)
0619 12:05:27.414893 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:27.414908 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:27.414999 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:27.415201 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:27.415203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:27.415487 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:27.415560 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:27.415563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.416376 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.416377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:27.417075 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:27.417078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.417520 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.417521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:27.417577 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:27.421001 (+ 3424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:27.421007 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:27.423113 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
W20260619 12:05:27.423336 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:27.414846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:45237 (local address 127.0.203.190:33023)
0619 12:05:27.415003 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:27.415005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:27.415035 (+ 30us) server_negotiation.cc:407] Connection header received
0619 12:05:27.415062 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:27.415062 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:27.415074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:27.415191 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:27.415689 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.416273 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.417189 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.417414 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.417492 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:27.423255 (+ 5763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:45237: BlockingRecv error: recv got EOF from 127.0.203.129:45237 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
Jun 19 12:05:27 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:27 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:27.546638 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:27.541685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:40923)
0619 12:05:27.541915 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:27.541928 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:27.542025 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:27.542191 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:27.542192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:27.542362 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:27.542415 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:27.542417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.543199 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.543200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:27.543828 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:27.543831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.544265 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.544266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:27.544311 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:27.544814 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:27.544818 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:27.546550 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":116,"threads_started":1}
W20260619 12:05:27.546872 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:27.541845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:40923 (local address 127.0.203.190:33023)
0619 12:05:27.541918 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:27.541921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:27.542015 (+ 94us) server_negotiation.cc:407] Connection header received
0619 12:05:27.542091 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:27.542092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:27.542106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:27.542171 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:27.542518 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.543096 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.543945 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.544177 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.544256 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:27.546783 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:40923: BlockingRecv error: recv got EOF from 127.0.203.130:40923 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 12:05:27 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:27 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.131@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:27.693682 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:27.688576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.131:52597)
0619 12:05:27.688754 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:27.688768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:27.688862 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:27.689024 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:27.689025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:27.689237 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:27.689310 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:27.689312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.690053 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.690054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:27.690653 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:27.690655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.691046 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.691047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:27.691100 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:27.691679 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:27.691685 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:27.693609 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
W20260619 12:05:27.693837 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:27.688709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:52597 (local address 127.0.203.190:33023)
0619 12:05:27.688771 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:27.688773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:27.688853 (+ 80us) server_negotiation.cc:407] Connection header received
0619 12:05:27.688938 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:27.688939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:27.688953 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:27.689003 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:27.689431 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.689961 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.690743 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:27.690953 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:27.691027 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:27.693765 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.131:52597: BlockingRecv error: recv got EOF from 127.0.203.131:52597 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 12:05:28 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:28 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.129@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:28.429143 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:28.423826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.129:58821)
0619 12:05:28.423958 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:28.423969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:28.424021 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:28.424229 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:28.424230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:28.424404 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:28.424464 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:28.424466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.425207 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.425209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:28.425846 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:28.425849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.426325 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.426326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:28.426382 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:28.426974 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:28.426979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:28.429056 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
W20260619 12:05:28.429294 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:28.423909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:58821 (local address 127.0.203.190:33023)
0619 12:05:28.424046 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:28.424048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:28.424056 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:05:28.424081 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:28.424082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:28.424094 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:28.424172 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:28.424543 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.425101 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.425956 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.426189 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.426289 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:28.429193 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.129:58821: BlockingRecv error: recv got EOF from 127.0.203.129:58821 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
Jun 19 12:05:28 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:05:28 dist-test-slave-44vj krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.203.130@KRBTEST.COM for oryx/127.0.203.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:05:28.552311 10028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:28.547309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33023 (local address 127.0.203.130:53581)
0619 12:05:28.547445 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:28.547459 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:28.547508 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:28.547680 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:28.547680 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:28.547836 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:28.547887 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:28.547889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.548609 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.548610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:28.549170 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:28.549173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.549568 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.549569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:28.549614 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:28.550134 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:28.550139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:28.552209 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.203.190:33023: Server oryx/127.0.203.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
W20260619 12:05:28.552412 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:28.547420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:53581 (local address 127.0.203.190:33023)
0619 12:05:28.547480 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:28.547482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:28.547534 (+ 52us) server_negotiation.cc:407] Connection header received
0619 12:05:28.547575 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:28.547576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:28.547589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:28.547666 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:28.547956 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.548509 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.549285 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.549491 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.549557 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:28.552346 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.203.130:53581: BlockingRecv error: recv got EOF from 127.0.203.130:53581 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:05:28.598652 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 9551
I20260619 12:05:28.603147 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 9703
I20260619 12:05:28.607569 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 9836
I20260619 12:05:28.613407 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 9481
2026-06-19T12:05:28Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17542 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T12:05:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:28Z Disabled control of system clock
I20260619 12:05:28.643002 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34083
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:34083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 12:05:28.733695 10032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:28.733966 10032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:28.733987 10032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:28.735330 10032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:28.735417 10032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:28.735430 10032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:28.735441 10032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:28.735455 10032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:28.736984 10032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:34083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:28.737285 10032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:28.737630 10032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:28.743132 10037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:28.743206 10032 server_base.cc:1034] running on GCE node
W20260619 12:05:28.743155 10038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:28.743166 10040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:28.743788 10032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:28.744123 10032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:28.745291 10032 hybrid_clock.cc:648] HybridClock initialized: now 1781870728745291 us; error 45 us; skew 500 ppm
I20260619 12:05:28.746822 10032 webserver.cc:460] Webserver started at http://127.0.203.190:34115/ using document root <none> and password file <none>
I20260619 12:05:28.747087 10032 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:28.747138 10032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:28.747287 10032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:28.748564 10032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7"
format_stamp: "Formatted at 2026-06-19 12:05:28 on dist-test-slave-44vj"
server_key: "78b432be67025cb190225483d115ab09"
server_key_iv: "766105542f52aca46664bbfc91b7d7e1"
server_key_version: "encryptionkey@0"
I20260619 12:05:28.748929 10032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7"
format_stamp: "Formatted at 2026-06-19 12:05:28 on dist-test-slave-44vj"
server_key: "78b432be67025cb190225483d115ab09"
server_key_iv: "766105542f52aca46664bbfc91b7d7e1"
server_key_version: "encryptionkey@0"
I20260619 12:05:28.750391 10032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:05:28.751425 10046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:28.751684 10032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:28.751788 10032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7"
format_stamp: "Formatted at 2026-06-19 12:05:28 on dist-test-slave-44vj"
server_key: "78b432be67025cb190225483d115ab09"
server_key_iv: "766105542f52aca46664bbfc91b7d7e1"
server_key_version: "encryptionkey@0"
I20260619 12:05:28.751860 10032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:28.763051 10032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:28.763469 10032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:28.763619 10032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:28.768572 10032 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:34083
I20260619 12:05:28.768646 10098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:34083 every 8 connection(s)
I20260619 12:05:28.769039 10032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:05:28.770040 10099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:28.773041 10099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7: Bootstrap starting.
I20260619 12:05:28.773710 10099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:28.774003 10099 log.cc:824] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:28.774951 10099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7: No bootstrap required, opened a new log
I20260619 12:05:28.776736 10099 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34083 } }
I20260619 12:05:28.776902 10099 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:28.776927 10099 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff0b6c27ac184a3d8cc6e7489dedb3a7, State: Initialized, Role: FOLLOWER
I20260619 12:05:28.777069 10099 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34083 } }
I20260619 12:05:28.777158 10099 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:05:28.777199 10099 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:05:28.777256 10099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:28.777866 10099 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34083 } }
I20260619 12:05:28.777963 10099 leader_election.cc:302] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff0b6c27ac184a3d8cc6e7489dedb3a7; no voters:
I20260619 12:05:28.778164 10099 leader_election.cc:288] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:05:28.778265 10102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:28.778470 10099 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:28.778606 10102 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [term 1 LEADER]: Becoming Leader. State: Replica: ff0b6c27ac184a3d8cc6e7489dedb3a7, State: Running, Role: LEADER
I20260619 12:05:28.778800 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10032
I20260619 12:05:28.778795 10102 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34083 } }
I20260619 12:05:28.778892 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:05:28.779117 814 external_mini_cluster.cc:1428] Setting key 529e18944d28769bba087ea9fb3f8123
I20260619 12:05:28.779551 10103 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34083 } } }
I20260619 12:05:28.779661 10103 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:28.779847 10102 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff0b6c27ac184a3d8cc6e7489dedb3a7. Latest consensus state: current_term: 1 leader_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0b6c27ac184a3d8cc6e7489dedb3a7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34083 } } }
I20260619 12:05:28.780157 10102 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:28.780498 10112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:28.781179 10112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:28.783283 10112 catalog_manager.cc:1347] Generated new cluster ID: dc24e1c81ee245af90c1c8890edd30f9
I20260619 12:05:28.783385 10112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:05:28.785873 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:28.779746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.0.203.190:34083)
0619 12:05:28.780460 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:28.780473 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:28.780512 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:05:28.780678 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:28.780688 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:28.780782 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:28.780936 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:28.781198 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.782640 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.783682 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.784199 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.784333 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:28.785044 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:28.785049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:28.785050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:28.785275 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:28.785316 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:28.785330 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:28.785390 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:28.785463 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 12:05:28.785654 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":73,"threads_started":1}
I20260619 12:05:28.797123 10112 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:05:28.797806 10112 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:28.805524 10112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff0b6c27ac184a3d8cc6e7489dedb3a7: Generated new TSK 0
I20260619 12:05:28.805848 10112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:05:28.813324 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34083
--builtin_ntp_servers=127.0.203.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 12:05:28.901994 10123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:28.902251 10123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:28.902272 10123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:28.903595 10123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:28.903647 10123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:28.903672 10123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:28.905076 10123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:28.905355 10123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:28.905704 10123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:28.911392 10129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:28.911428 10128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:28.911496 10131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:28.911991 10123 server_base.cc:1034] running on GCE node
I20260619 12:05:28.912259 10123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:28.912537 10123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:28.913720 10123 hybrid_clock.cc:648] HybridClock initialized: now 1781870728913689 us; error 63 us; skew 500 ppm
I20260619 12:05:28.915303 10123 webserver.cc:460] Webserver started at http://127.0.203.129:43097/ using document root <none> and password file <none>
I20260619 12:05:28.915627 10123 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:28.915693 10123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:28.915838 10123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:28.917109 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "a2fb4725ec774b6f9076b8c109c57f06"
format_stamp: "Formatted at 2026-06-19 12:05:28 on dist-test-slave-44vj"
server_key: "51880f169faa4f6d280b4c729e956a23"
server_key_iv: "4252442fca7f8fa3a95dbf55c463e5bc"
server_key_version: "encryptionkey@0"
I20260619 12:05:28.917484 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "a2fb4725ec774b6f9076b8c109c57f06"
format_stamp: "Formatted at 2026-06-19 12:05:28 on dist-test-slave-44vj"
server_key: "51880f169faa4f6d280b4c729e956a23"
server_key_iv: "4252442fca7f8fa3a95dbf55c463e5bc"
server_key_version: "encryptionkey@0"
I20260619 12:05:28.919030 10123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:05:28.920107 10137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:28.920459 10123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:28.920578 10123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "a2fb4725ec774b6f9076b8c109c57f06"
format_stamp: "Formatted at 2026-06-19 12:05:28 on dist-test-slave-44vj"
server_key: "51880f169faa4f6d280b4c729e956a23"
server_key_iv: "4252442fca7f8fa3a95dbf55c463e5bc"
server_key_version: "encryptionkey@0"
I20260619 12:05:28.920678 10123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:28.933059 10123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:28.933475 10123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:28.933619 10123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:28.933895 10123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:28.934270 10123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:28.934305 10123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:28.934337 10123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:28.934355 10123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:28.941360 10123 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:42121
I20260619 12:05:28.941452 10250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:42121 every 8 connection(s)
I20260619 12:05:28.941787 10123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:05:28.947556 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:28.942578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:41801 (local address 127.0.203.190:34083)
0619 12:05:28.942637 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:28.942639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:28.942960 (+ 321us) server_negotiation.cc:407] Connection header received
0619 12:05:28.943564 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:28.943566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:28.943581 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:28.943650 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:28.944321 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.944879 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.946041 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.946295 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.946362 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:28.947226 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:28.947228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:28.947229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:28.947241 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:28.947262 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:28.947266 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:28.947323 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:28.947449 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 12:05:28.947482 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:05:28.947743 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:28.942567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34083 (local address 127.0.203.129:41801)
0619 12:05:28.942789 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:28.942825 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:28.943444 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:28.943707 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:28.943711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:28.943721 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:28.944240 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:28.944244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.945032 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.945033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:28.945891 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:28.945894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:28.946374 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:28.946374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:28.946483 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:28.947043 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:05:28.947043 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:28.947074 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:05:28.947104 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:28.947334 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:28.947338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:28.947340 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:05:28.947438 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:05:28.947534 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
I20260619 12:05:28.948432 10251 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34083
I20260619 12:05:28.948541 10251 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:28.948746 10251 heartbeater.cc:511] Master 127.0.203.190:34083 requested a full tablet report, sending...
I20260619 12:05:28.949491 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10123
I20260619 12:05:28.949486 10063 ts_manager.cc:194] Registered new tserver with Master: a2fb4725ec774b6f9076b8c109c57f06 (127.0.203.129:42121)
I20260619 12:05:28.949584 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:05:28.949807 814 external_mini_cluster.cc:1428] Setting key 7ba2253cb580654702216658b4bf4009
I20260619 12:05:28.950436 10063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.129:41801
I20260619 12:05:28.951026 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34083
--builtin_ntp_servers=127.0.203.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 12:05:29.038451 10254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:29.038679 10254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:29.038703 10254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:29.039994 10254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:29.040055 10254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:29.040081 10254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:29.041543 10254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:29.041816 10254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:29.042147 10254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:29.047351 10259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.047385 10260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.047343 10262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.047843 10254 server_base.cc:1034] running on GCE node
I20260619 12:05:29.048079 10254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:29.048331 10254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:29.049510 10254 hybrid_clock.cc:648] HybridClock initialized: now 1781870729049495 us; error 27 us; skew 500 ppm
I20260619 12:05:29.050999 10254 webserver.cc:460] Webserver started at http://127.0.203.130:40085/ using document root <none> and password file <none>
I20260619 12:05:29.051249 10254 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:29.051306 10254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:29.051585 10254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:29.052987 10254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "938650742cfe4ce681f2256649b16576"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "493c808c2766af2b9b19aefc8b795026"
server_key_iv: "8457905e51cf21e169bd9196c75e1cec"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.053345 10254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "938650742cfe4ce681f2256649b16576"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "493c808c2766af2b9b19aefc8b795026"
server_key_iv: "8457905e51cf21e169bd9196c75e1cec"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.054841 10254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:05:29.055799 10268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.056026 10254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:29.056134 10254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "938650742cfe4ce681f2256649b16576"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "493c808c2766af2b9b19aefc8b795026"
server_key_iv: "8457905e51cf21e169bd9196c75e1cec"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.056205 10254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:29.077983 10254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:29.078323 10254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:29.078457 10254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:29.078702 10254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:29.079150 10254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:29.079195 10254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.079226 10254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:29.079246 10254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.086097 10254 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:32843
I20260619 12:05:29.086159 10381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:32843 every 8 connection(s)
I20260619 12:05:29.086515 10254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:29.086938 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10254
I20260619 12:05:29.087064 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:05:29.087313 814 external_mini_cluster.cc:1428] Setting key 6316aaa60d4c8501b13384d6a1537a0c
I20260619 12:05:29.088526 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34083
--builtin_ntp_servers=127.0.203.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 12:05:29.093199 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.087614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:40643 (local address 127.0.203.190:34083)
0619 12:05:29.087705 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.087708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.088084 (+ 376us) server_negotiation.cc:407] Connection header received
0619 12:05:29.088860 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.088861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.088877 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.088933 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.090058 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.090570 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.091770 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.091981 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.092039 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:29.092907 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:29.092908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:29.092909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:29.092915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:29.092932 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:29.092937 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:29.092989 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:29.093075 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 12:05:29.093108 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:05:29.093410 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.087598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34083 (local address 127.0.203.130:40643)
0619 12:05:29.087863 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:29.087910 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:29.088683 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:29.089056 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:29.089063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:29.089078 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:29.089905 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:29.089913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.090726 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.090728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:29.091674 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:29.091678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.092061 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.092062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:29.092202 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:29.092759 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:05:29.092760 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:29.092790 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:05:29.092816 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:29.092998 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:29.093003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:29.093007 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:29.093073 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 12:05:29.093189 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
I20260619 12:05:29.093847 10382 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34083
I20260619 12:05:29.093948 10382 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:29.094146 10382 heartbeater.cc:511] Master 127.0.203.190:34083 requested a full tablet report, sending...
I20260619 12:05:29.094494 10063 ts_manager.cc:194] Registered new tserver with Master: 938650742cfe4ce681f2256649b16576 (127.0.203.130:32843)
I20260619 12:05:29.094878 10063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.130:40643
W20260619 12:05:29.180751 10385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:29.181025 10385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:29.181056 10385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:29.182399 10385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:29.182449 10385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:29.182487 10385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:29.184015 10385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:29.184316 10385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:29.184669 10385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:29.190256 10391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.190272 10390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.190515 10385 server_base.cc:1034] running on GCE node
W20260619 12:05:29.190276 10393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.190830 10385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:29.191113 10385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:29.192282 10385 hybrid_clock.cc:648] HybridClock initialized: now 1781870729192256 us; error 39 us; skew 500 ppm
I20260619 12:05:29.193605 10385 webserver.cc:460] Webserver started at http://127.0.203.131:41037/ using document root <none> and password file <none>
I20260619 12:05:29.193826 10385 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:29.193872 10385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:29.193989 10385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:29.195161 10385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "332e9d09f380421683a6e516483b11a6"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "bfe81e26823a0d92076884141036ea06"
server_key_iv: "c4899a29f1258f4485fa37937d03198a"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.195562 10385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "332e9d09f380421683a6e516483b11a6"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "bfe81e26823a0d92076884141036ea06"
server_key_iv: "c4899a29f1258f4485fa37937d03198a"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.196990 10385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:29.197777 10399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.197952 10385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:29.198065 10385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "332e9d09f380421683a6e516483b11a6"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "bfe81e26823a0d92076884141036ea06"
server_key_iv: "c4899a29f1258f4485fa37937d03198a"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.198134 10385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:29.210301 10385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:29.210659 10385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:29.210801 10385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:29.211050 10385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:29.211447 10385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:29.211496 10385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.211527 10385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:29.211550 10385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.218037 10385 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:41283
I20260619 12:05:29.218179 10512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:41283 every 8 connection(s)
I20260619 12:05:29.218449 10385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:05:29.224738 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.219407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:52277 (local address 127.0.203.190:34083)
0619 12:05:29.219495 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.219498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.219665 (+ 167us) server_negotiation.cc:407] Connection header received
0619 12:05:29.220350 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.220351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.220363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.220466 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.221344 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.221874 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.223053 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.223324 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.223471 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:29.224480 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:29.224481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:29.224482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:29.224488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:29.224507 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:29.224512 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:29.224568 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:29.224642 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 12:05:29.224670 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:05:29.224996 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.219259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34083 (local address 127.0.203.131:52277)
0619 12:05:29.219497 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:29.219549 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:29.220192 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:29.220544 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:29.220548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:29.220559 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:29.221186 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:29.221192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.221975 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.221976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:29.222947 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:29.222950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:29.223478 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:29.223479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:29.223610 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:29.224321 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:05:29.224322 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:29.224362 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:05:29.224391 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:29.224575 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:29.224578 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:29.224581 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:05:29.224635 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:05:29.224734 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260619 12:05:29.225054 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10385
I20260619 12:05:29.225135 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:05:29.225366 814 external_mini_cluster.cc:1428] Setting key 95c2340ca81027b82d42ae3e3a1cc02c
I20260619 12:05:29.225724 10513 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34083
I20260619 12:05:29.225836 10513 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:29.226053 10513 heartbeater.cc:511] Master 127.0.203.190:34083 requested a full tablet report, sending...
I20260619 12:05:29.226434 10063 ts_manager.cc:194] Registered new tserver with Master: 332e9d09f380421683a6e516483b11a6 (127.0.203.131:41283)
I20260619 12:05:29.226836 10063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.131:52277
I20260619 12:05:29.237771 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:05:29.239961 10113 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:29.239502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.0.203.190:34083)
0619 12:05:29.239569 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.239571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.239627 (+ 56us) server_negotiation.cc:407] Connection header received
0619 12:05:29.239679 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.239679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.239692 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.239756 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.239886 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35466: BlockingRecv error: recv got EOF from 127.0.0.1:35466 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:05:29.240036 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10123
I20260619 12:05:29.244966 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10254
I20260619 12:05:29.249300 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10385
I20260619 12:05:29.253980 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10032
2026-06-19T12:05:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (639 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T12:05:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:29Z Disabled control of system clock
I20260619 12:05:29.282116 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41937
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:41937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:05:29.373673 10524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:29.373925 10524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:29.373951 10524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:29.375293 10524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:29.375347 10524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:29.375403 10524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:29.375429 10524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:29.375447 10524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:29.377087 10524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:41937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:29.377341 10524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:29.377619 10524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:29.382906 10530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.382876 10532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.382930 10529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.383070 10524 server_base.cc:1034] running on GCE node
I20260619 12:05:29.383554 10524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:29.383857 10524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:29.385020 10524 hybrid_clock.cc:648] HybridClock initialized: now 1781870729385009 us; error 36 us; skew 500 ppm
I20260619 12:05:29.386485 10524 webserver.cc:460] Webserver started at http://127.0.203.190:45975/ using document root <none> and password file <none>
I20260619 12:05:29.386732 10524 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:29.386786 10524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:29.386922 10524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:29.388094 10524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "a08df2a1f0fc4565a0b5547b689da36a"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "4e4db26978fb3f91bb405896d0d6d116"
server_key_iv: "b131036cc8a967f0355499caeefa05cf"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.388442 10524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "a08df2a1f0fc4565a0b5547b689da36a"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "4e4db26978fb3f91bb405896d0d6d116"
server_key_iv: "b131036cc8a967f0355499caeefa05cf"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.389807 10524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:05:29.390637 10538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.390861 10524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:29.390960 10524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "a08df2a1f0fc4565a0b5547b689da36a"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "4e4db26978fb3f91bb405896d0d6d116"
server_key_iv: "b131036cc8a967f0355499caeefa05cf"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.391026 10524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:29.394541 10524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:29.394845 10524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:29.394976 10524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:29.399168 10524 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:41937
I20260619 12:05:29.399293 10590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:41937 every 8 connection(s)
I20260619 12:05:29.399631 10524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:05:29.400504 10591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:29.403826 10591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a: Bootstrap starting.
I20260619 12:05:29.404511 10591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:29.404816 10591 log.cc:824] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:29.405756 10591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a: No bootstrap required, opened a new log
I20260619 12:05:29.407496 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10524
I20260619 12:05:29.407613 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:05:29.407793 814 external_mini_cluster.cc:1428] Setting key 6467984352d115bb916a72bcfafcfb3c
I20260619 12:05:29.407648 10591 raft_consensus.cc:348] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08df2a1f0fc4565a0b5547b689da36a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41937 } }
I20260619 12:05:29.407830 10591 raft_consensus.cc:374] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:29.407859 10591 raft_consensus.cc:729] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08df2a1f0fc4565a0b5547b689da36a, State: Initialized, Role: FOLLOWER
I20260619 12:05:29.408046 10591 consensus_queue.cc:260] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08df2a1f0fc4565a0b5547b689da36a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41937 } }
I20260619 12:05:29.408121 10591 raft_consensus.cc:388] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:05:29.408152 10591 raft_consensus.cc:482] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:05:29.408203 10591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:29.408967 10591 raft_consensus.cc:504] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08df2a1f0fc4565a0b5547b689da36a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41937 } }
I20260619 12:05:29.409085 10591 leader_election.cc:302] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a08df2a1f0fc4565a0b5547b689da36a; no voters:
I20260619 12:05:29.409267 10591 leader_election.cc:288] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:05:29.409588 10591 sys_catalog.cc:564] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:29.409726 10596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:29.409878 10596 raft_consensus.cc:686] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [term 1 LEADER]: Becoming Leader. State: Replica: a08df2a1f0fc4565a0b5547b689da36a, State: Running, Role: LEADER
I20260619 12:05:29.409976 10596 consensus_queue.cc:237] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08df2a1f0fc4565a0b5547b689da36a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41937 } }
I20260619 12:05:29.410413 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.408352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.0.203.190:41937)
0619 12:05:29.408664 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.408677 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.408723 (+ 46us) server_negotiation.cc:407] Connection header received
0619 12:05:29.408860 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.408866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.408909 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.409040 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.409817 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:29.409821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:29.409821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:29.409827 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:29.409877 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:29.409894 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:29.409952 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:29.410109 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 12:05:29.410190 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":118,"threads_started":1}
I20260619 12:05:29.411655 10597 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a08df2a1f0fc4565a0b5547b689da36a. Latest consensus state: current_term: 1 leader_uuid: "a08df2a1f0fc4565a0b5547b689da36a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08df2a1f0fc4565a0b5547b689da36a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41937 } } }
I20260619 12:05:29.411801 10597 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:29.411877 10598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a08df2a1f0fc4565a0b5547b689da36a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08df2a1f0fc4565a0b5547b689da36a" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41937 } } }
I20260619 12:05:29.412024 10598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a [sys.catalog]: This master's current role is: LEADER
W20260619 12:05:29.412724 10610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:05:29.412793 10610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:05:29.412840 10611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:29.413015 10611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:29.414723 10611 catalog_manager.cc:1347] Generated new cluster ID: 7327e330e58647779346fc2a9b2e2b6b
I20260619 12:05:29.414814 10611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:05:29.428118 10611 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:05:29.428342 10611 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:29.432960 10611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a08df2a1f0fc4565a0b5547b689da36a: Generated new TSK 0
I20260619 12:05:29.433211 10611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:05:29.436069 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41937
--builtin_ntp_servers=127.0.203.148:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:05:29.527040 10615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:29.527284 10615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:29.527318 10615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:29.528616 10615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:29.528666 10615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:29.528703 10615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:29.530118 10615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:29.530393 10615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:29.530718 10615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:29.537333 10623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.537503 10621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.537396 10620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.537762 10615 server_base.cc:1034] running on GCE node
I20260619 12:05:29.538025 10615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:29.538350 10615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:29.539520 10615 hybrid_clock.cc:648] HybridClock initialized: now 1781870729539503 us; error 31 us; skew 500 ppm
I20260619 12:05:29.541241 10615 webserver.cc:460] Webserver started at http://127.0.203.129:40459/ using document root <none> and password file <none>
I20260619 12:05:29.541561 10615 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:29.541644 10615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:29.541818 10615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:29.543396 10615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "1dbea6cd92274c3b965bf9ff22d0b104"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "b88b23a9e58ca8a004f12af6c9aa27d4"
server_key_iv: "2621d0a00a720ad13a8613d90cda2806"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.543833 10615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "1dbea6cd92274c3b965bf9ff22d0b104"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "b88b23a9e58ca8a004f12af6c9aa27d4"
server_key_iv: "2621d0a00a720ad13a8613d90cda2806"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.545667 10615 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260619 12:05:29.546613 10629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.546823 10615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:29.546937 10615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "1dbea6cd92274c3b965bf9ff22d0b104"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "b88b23a9e58ca8a004f12af6c9aa27d4"
server_key_iv: "2621d0a00a720ad13a8613d90cda2806"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.547019 10615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:29.552414 10615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:29.552767 10615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:29.552914 10615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:29.553157 10615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:29.553565 10615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:29.553617 10615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.553645 10615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:29.553660 10615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.560348 10615 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:35057
I20260619 12:05:29.560431 10742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:35057 every 8 connection(s)
I20260619 12:05:29.560784 10615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:05:29.562081 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10615
I20260619 12:05:29.562171 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:05:29.562366 814 external_mini_cluster.cc:1428] Setting key 92a10983cfa6828a2edb00dce3800dfe
I20260619 12:05:29.563433 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41937
--builtin_ntp_servers=127.0.203.148:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 12:05:29.564697 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.561659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:52047 (local address 127.0.203.190:41937)
0619 12:05:29.561727 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.561730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.562149 (+ 419us) server_negotiation.cc:407] Connection header received
0619 12:05:29.562861 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.562862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.562870 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.562964 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.564325 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:29.564327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:29.564327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:29.564333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:29.564356 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:29.564362 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:29.564439 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:29.564607 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 12:05:29.564626 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:05:29.565006 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.561633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41937 (local address 127.0.203.129:52047)
0619 12:05:29.561971 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:29.562027 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:29.562760 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:29.563057 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:29.563062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:29.563075 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:29.564145 (+ 1070us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:05:29.564146 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:29.564183 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:05:29.564218 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:29.564486 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:29.564499 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:29.564504 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:05:29.564623 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 12:05:29.564715 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":188,"threads_started":1}
I20260619 12:05:29.565524 10743 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41937
I20260619 12:05:29.565640 10743 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:29.565814 10743 heartbeater.cc:511] Master 127.0.203.190:41937 requested a full tablet report, sending...
I20260619 12:05:29.566422 10555 ts_manager.cc:194] Registered new tserver with Master: 1dbea6cd92274c3b965bf9ff22d0b104 (127.0.203.129:35057)
W20260619 12:05:29.655414 10746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:29.655676 10746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:29.655702 10746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:29.657001 10746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:29.657042 10746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:29.657068 10746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:29.658459 10746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:29.658753 10746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:29.659099 10746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:29.664832 10752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.664958 10754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.665051 10746 server_base.cc:1034] running on GCE node
W20260619 12:05:29.664845 10751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.665413 10746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:29.665721 10746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:29.666883 10746 hybrid_clock.cc:648] HybridClock initialized: now 1781870729666866 us; error 51 us; skew 500 ppm
I20260619 12:05:29.668344 10746 webserver.cc:460] Webserver started at http://127.0.203.130:35607/ using document root <none> and password file <none>
I20260619 12:05:29.668599 10746 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:29.668650 10746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:29.668820 10746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:29.670082 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "187bd0a090df463cb1945e04720cca9f"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "39f9f38d4d390c90b6168185886924df"
server_key_iv: "f0bd08699490fd785e69a49c522c5325"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.670457 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "187bd0a090df463cb1945e04720cca9f"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "39f9f38d4d390c90b6168185886924df"
server_key_iv: "f0bd08699490fd785e69a49c522c5325"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.672075 10746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:05:29.673013 10760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.673243 10746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:29.673360 10746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "187bd0a090df463cb1945e04720cca9f"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "39f9f38d4d390c90b6168185886924df"
server_key_iv: "f0bd08699490fd785e69a49c522c5325"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.673434 10746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:29.677141 10746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:29.677439 10746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:29.677563 10746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:29.677771 10746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:29.678098 10746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:29.678133 10746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.678159 10746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:29.678184 10746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.685186 10746 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:41491
I20260619 12:05:29.685238 10873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:41491 every 8 connection(s)
I20260619 12:05:29.685631 10746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:29.689035 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.686512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:41679 (local address 127.0.203.190:41937)
0619 12:05:29.686563 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.686565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.686841 (+ 276us) server_negotiation.cc:407] Connection header received
0619 12:05:29.687401 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.687402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.687409 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.687460 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.688760 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:29.688761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:29.688762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:29.688766 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:29.688783 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:29.688787 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:29.688840 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:29.688913 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 12:05:29.688927 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:05:29.689234 10876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.686491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41937 (local address 127.0.203.130:41679)
0619 12:05:29.686698 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:29.686738 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:29.687283 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:29.687544 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:29.687548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:29.687558 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:29.688529 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:05:29.688529 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:29.688612 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:05:29.688648 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:29.688841 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:29.688846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:29.688850 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:29.688921 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:05:29.688990 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260619 12:05:29.689247 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10746
I20260619 12:05:29.689366 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:05:29.689596 814 external_mini_cluster.cc:1428] Setting key 13d3d9a7671326ba9c3cabafa2430ef5
I20260619 12:05:29.690133 10874 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41937
I20260619 12:05:29.690263 10874 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:29.690495 10874 heartbeater.cc:511] Master 127.0.203.190:41937 requested a full tablet report, sending...
I20260619 12:05:29.690837 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41937
--builtin_ntp_servers=127.0.203.148:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 12:05:29.691000 10555 ts_manager.cc:194] Registered new tserver with Master: 187bd0a090df463cb1945e04720cca9f (127.0.203.130:41491)
W20260619 12:05:29.777456 10877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:29.777707 10877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:29.777740 10877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:29.779078 10877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:29.779131 10877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:29.779170 10877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:29.780757 10877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:29.781044 10877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:29.781342 10877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:29.786859 10885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.786859 10883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:29.786882 10882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:29.786904 10877 server_base.cc:1034] running on GCE node
I20260619 12:05:29.787344 10877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:29.787664 10877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:29.788801 10877 hybrid_clock.cc:648] HybridClock initialized: now 1781870729788781 us; error 45 us; skew 500 ppm
I20260619 12:05:29.790081 10877 webserver.cc:460] Webserver started at http://127.0.203.131:43695/ using document root <none> and password file <none>
I20260619 12:05:29.790334 10877 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:29.790387 10877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:29.790493 10877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:29.791641 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "ae17be5c5a4242f98ebb414784739f22"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "53b7b7dbbbe3b701a0525a74a5bc8f6e"
server_key_iv: "b9fb983110564cb31d8ca55ed4469d3d"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.791973 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "ae17be5c5a4242f98ebb414784739f22"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "53b7b7dbbbe3b701a0525a74a5bc8f6e"
server_key_iv: "b9fb983110564cb31d8ca55ed4469d3d"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.793381 10877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 12:05:29.794255 10891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.794457 10877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:29.794574 10877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "ae17be5c5a4242f98ebb414784739f22"
format_stamp: "Formatted at 2026-06-19 12:05:29 on dist-test-slave-44vj"
server_key: "53b7b7dbbbe3b701a0525a74a5bc8f6e"
server_key_iv: "b9fb983110564cb31d8ca55ed4469d3d"
server_key_version: "encryptionkey@0"
I20260619 12:05:29.794659 10877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:29.798199 10877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:29.798472 10877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:29.798588 10877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:29.798766 10877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:29.799064 10877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:29.799099 10877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.799119 10877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:29.799132 10877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:29.805914 10877 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:35527
I20260619 12:05:29.805967 11004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:35527 every 8 connection(s)
I20260619 12:05:29.806311 10877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:05:29.806717 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 10877
I20260619 12:05:29.806838 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:05:29.807062 814 external_mini_cluster.cc:1428] Setting key 799d9df191c99d2b8a78705e8f96a544
I20260619 12:05:29.810184 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.807239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:48315 (local address 127.0.203.190:41937)
0619 12:05:29.807449 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.807451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.807857 (+ 406us) server_negotiation.cc:407] Connection header received
0619 12:05:29.808423 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.808423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.808430 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.808484 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.809923 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:29.809924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:29.809925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:05:29.809930 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:29.809950 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:05:29.809955 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:29.810014 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:29.810091 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 12:05:29.810107 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260619 12:05:29.810365 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:29.807279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41937 (local address 127.0.203.131:48315)
0619 12:05:29.807707 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:29.807764 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:29.808330 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:29.808554 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:29.808557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:29.808567 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:29.809722 (+ 1155us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:05:29.809723 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:29.809757 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:05:29.809820 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:29.810022 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:29.810025 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:29.810027 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:05:29.810091 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:05:29.810149 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":151,"threads_started":1}
I20260619 12:05:29.811075 11005 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41937
I20260619 12:05:29.811197 11005 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:29.811443 11005 heartbeater.cc:511] Master 127.0.203.190:41937 requested a full tablet report, sending...
I20260619 12:05:29.811867 10555 ts_manager.cc:194] Registered new tserver with Master: ae17be5c5a4242f98ebb414784739f22 (127.0.203.131:35527)
I20260619 12:05:29.819531 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:05:29.821755 10595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:05:29.821194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.0.203.190:41937)
0619 12:05:29.821272 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:29.821275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:29.821371 (+ 96us) server_negotiation.cc:407] Connection header received
0619 12:05:29.821439 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:29.821439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:29.821444 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:29.821511 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:29.821661 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42358: BlockingRecv error: recv got EOF from 127.0.0.1:42358 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:05:29.821866 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10615
I20260619 12:05:29.826896 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10746
I20260619 12:05:29.831550 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10877
I20260619 12:05:29.836115 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 10524
2026-06-19T12:05:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (581 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:05:29 dist-test-slave-44vj krb5kdc[11015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:05:29 dist-test-slave-44vj krb5kdc[11015](info): set up 2 sockets
Jun 19 12:05:29 dist-test-slave-44vj krb5kdc[11015](info): commencing operation
krb5kdc: starting...
W20260619 12:05:30.804565 814 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:05:30 dist-test-slave-44vj krb5kdc[11015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:05:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:05:30.931068 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42041
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:42041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:31.020473 11028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:31.020749 11028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:31.020785 11028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:31.022140 11028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:05:31.022186 11028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:31.022205 11028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:31.022221 11028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:05:31.022236 11028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:05:31.023998 11028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:42041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:31.024287 11028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:31.024525 11028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:31.029834 11036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:31.030041 11028 server_base.cc:1034] running on GCE node
W20260619 12:05:31.029910 11033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:31.029826 11034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:31.030485 11028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:31.030799 11028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:31.031965 11028 hybrid_clock.cc:648] HybridClock initialized: now 1781870731031943 us; error 38 us; skew 500 ppm
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:31.034628 11028 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:05:31.035135 11028 webserver.cc:460] Webserver started at http://127.0.203.190:37733/ using document root <none> and password file <none>
I20260619 12:05:31.035348 11028 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:31.035434 11028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:31.035547 11028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:31.036664 11028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "545cc94b73554369a73f18447e7a1da7"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "4e94e67b346715ff4bfc813ac1da07ea"
server_key_iv: "df7c15fbc0667bd6abca0899b59a3661"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.036993 11028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "545cc94b73554369a73f18447e7a1da7"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "4e94e67b346715ff4bfc813ac1da07ea"
server_key_iv: "df7c15fbc0667bd6abca0899b59a3661"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.038552 11028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:05:31.039700 11043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.040061 11028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:31.040196 11028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "545cc94b73554369a73f18447e7a1da7"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "4e94e67b346715ff4bfc813ac1da07ea"
server_key_iv: "df7c15fbc0667bd6abca0899b59a3661"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.040333 11028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:31.061650 11028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:31.064025 11028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:31.064150 11028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:31.068590 11028 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:42041
I20260619 12:05:31.068661 11095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:42041 every 8 connection(s)
I20260619 12:05:31.069044 11028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:05:31.070200 11096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:31.073123 11096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7: Bootstrap starting.
I20260619 12:05:31.073830 11096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:31.074138 11096 log.cc:824] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:31.075179 11096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7: No bootstrap required, opened a new log
I20260619 12:05:31.077217 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 11028
I20260619 12:05:31.077313 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:05:31.077257 11096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cc94b73554369a73f18447e7a1da7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42041 } }
I20260619 12:05:31.077498 814 external_mini_cluster.cc:1428] Setting key 64becc511e4d3fd561d6ab10ebf02dc0
I20260619 12:05:31.077487 11096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:31.077543 11096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545cc94b73554369a73f18447e7a1da7, State: Initialized, Role: FOLLOWER
I20260619 12:05:31.077735 11096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cc94b73554369a73f18447e7a1da7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42041 } }
I20260619 12:05:31.077836 11096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:05:31.077870 11096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:05:31.077925 11096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:31.079017 11096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cc94b73554369a73f18447e7a1da7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42041 } }
I20260619 12:05:31.079156 11096 leader_election.cc:302] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545cc94b73554369a73f18447e7a1da7; no voters:
I20260619 12:05:31.079394 11096 leader_election.cc:288] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:05:31.079486 11101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:31.079738 11101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [term 1 LEADER]: Becoming Leader. State: Replica: 545cc94b73554369a73f18447e7a1da7, State: Running, Role: LEADER
I20260619 12:05:31.079869 11096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:05:31.079931 11101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cc94b73554369a73f18447e7a1da7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42041 } }
I20260619 12:05:31.080636 11103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545cc94b73554369a73f18447e7a1da7. Latest consensus state: current_term: 1 leader_uuid: "545cc94b73554369a73f18447e7a1da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cc94b73554369a73f18447e7a1da7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42041 } } }
I20260619 12:05:31.080791 11103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:31.081300 11112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:05:31.081384 11102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545cc94b73554369a73f18447e7a1da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cc94b73554369a73f18447e7a1da7" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42041 } } }
I20260619 12:05:31.081457 11102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7 [sys.catalog]: This master's current role is: LEADER
I20260619 12:05:31.082068 11112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:05:31.084259 11112 catalog_manager.cc:1347] Generated new cluster ID: b4de37f9f2054a0591d015e17bd19e0e
I20260619 12:05:31.084326 11112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:05:31.091997 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.078148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45550 (local address 127.0.203.190:42041)
0619 12:05:31.078406 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.078420 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.078463 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:05:31.078982 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.078993 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.079088 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.079297 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:31.080227 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.081947 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.083085 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.083679 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.083796 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.087156 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:31.087163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:31.087164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:31.087183 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:31.089530 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.090063 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.090067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.090068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.090226 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.090442 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.090443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.090443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.090868 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:31.091057 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.091298 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.091526 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":131,"threads_started":1}
I20260619 12:05:31.095093 11112 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:05:31.095690 11112 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:05:31.102692 11112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7: Generated new TSK 0
I20260619 12:05:31.102885 11112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:05:31.135186 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42041
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:31.223757 11122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:31.224017 11122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:31.224045 11122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:31.225296 11122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:31.225354 11122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:31.225383 11122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:05:31.227722 11122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:31.227984 11122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:31.228204 11122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:31.233727 11128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:31.233733 11127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:31.233733 11130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:31.234210 11122 server_base.cc:1034] running on GCE node
I20260619 12:05:31.234450 11122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:31.234715 11122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:31.235889 11122 hybrid_clock.cc:648] HybridClock initialized: now 1781870731235868 us; error 37 us; skew 500 ppm
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:31.238478 11122 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:05:31.238965 11122 webserver.cc:460] Webserver started at http://127.0.203.129:44783/ using document root <none> and password file <none>
I20260619 12:05:31.239172 11122 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:31.239219 11122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:31.239312 11122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:31.240386 11122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "7306931add8945a2874c7e4479cac38c"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "d0552edecba387d4e93821298f8e1e98"
server_key_iv: "44e8b00f31f16b4ad1825ace69c1240b"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.240693 11122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "7306931add8945a2874c7e4479cac38c"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "d0552edecba387d4e93821298f8e1e98"
server_key_iv: "44e8b00f31f16b4ad1825ace69c1240b"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.242033 11122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:05:31.242844 11137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.243073 11122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:31.243167 11122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "7306931add8945a2874c7e4479cac38c"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "d0552edecba387d4e93821298f8e1e98"
server_key_iv: "44e8b00f31f16b4ad1825ace69c1240b"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.243256 11122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:31.262305 11122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:31.264590 11122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:31.264721 11122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:31.264977 11122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:31.265309 11122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:31.265355 11122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.265378 11122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:31.265389 11122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.272987 11122 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:45457
I20260619 12:05:31.273023 11250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:45457 every 8 connection(s)
I20260619 12:05:31.273590 11122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:05:31.281925 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 11122
I20260619 12:05:31.282027 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:05:31.282255 814 external_mini_cluster.cc:1428] Setting key fa7f04f4e189adfec3120b03a5a434b2
I20260619 12:05:31.286288 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.274658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:34827 (local address 127.0.203.190:42041)
0619 12:05:31.274767 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.274769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.274954 (+ 185us) server_negotiation.cc:407] Connection header received
0619 12:05:31.275492 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.275494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.275512 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.275613 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:31.276707 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.277315 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.278449 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.278668 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.278757 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.282574 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:31.282577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:31.282577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:31.282613 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:31.284304 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.284860 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.284862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.284862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.284951 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.285281 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.285282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.285282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.285451 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:31.285530 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.286054 (+ 524us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.286167 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 12:05:31.286398 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.274587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42041 (local address 127.0.203.129:34827)
0619 12:05:31.274767 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.274813 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.275326 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.275693 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.275697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.276344 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:31.276551 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.276556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.277452 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.277454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.278327 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.278329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.278807 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.278808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.278944 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.279642 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:31.279654 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:31.281633 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:31.284429 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:31.284442 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:31.284445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:31.284757 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:31.285056 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:31.285057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:31.285057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:31.285195 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:31.285574 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:31.285585 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:31.285872 (+ 287us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.286003 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.286179 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260619 12:05:31.287052 11251 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42041
I20260619 12:05:31.287159 11251 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:31.287387 11251 heartbeater.cc:511] Master 127.0.203.190:42041 requested a full tablet report, sending...
I20260619 12:05:31.287937 11060 ts_manager.cc:194] Registered new tserver with Master: 7306931add8945a2874c7e4479cac38c (127.0.203.129:45457)
I20260619 12:05:31.288574 11060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:34827
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:05:31.310043 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42041
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:31.401660 11256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:31.402000 11256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:31.402043 11256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:31.403618 11256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:31.403669 11256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:31.403702 11256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:05:31.405375 11256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:31.405651 11256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:31.405922 11256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:31.411513 11261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:31.411554 11264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:31.411513 11262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:31.411855 11256 server_base.cc:1034] running on GCE node
I20260619 12:05:31.412102 11256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:31.412379 11256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:31.413563 11256 hybrid_clock.cc:648] HybridClock initialized: now 1781870731413534 us; error 52 us; skew 500 ppm
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:31.416392 11256 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:05:31.416925 11256 webserver.cc:460] Webserver started at http://127.0.203.130:44509/ using document root <none> and password file <none>
I20260619 12:05:31.417167 11256 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:31.417217 11256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:31.417348 11256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:31.418454 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "5b34cf9b9d9b4f79938d78db414af376"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "20b6ccc4ad423e9677c02b47ce982f3f"
server_key_iv: "a2ba8d8439781da045da10ba2cb088e2"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.418815 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "5b34cf9b9d9b4f79938d78db414af376"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "20b6ccc4ad423e9677c02b47ce982f3f"
server_key_iv: "a2ba8d8439781da045da10ba2cb088e2"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.420351 11256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:05:31.421325 11271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.421635 11256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:05:31.421758 11256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "5b34cf9b9d9b4f79938d78db414af376"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "20b6ccc4ad423e9677c02b47ce982f3f"
server_key_iv: "a2ba8d8439781da045da10ba2cb088e2"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.421837 11256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:31.434448 11256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:31.437080 11256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:31.437240 11256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:31.437492 11256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:31.437827 11256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:31.437862 11256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.437892 11256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:31.437909 11256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.444653 11256 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:34425
I20260619 12:05:31.444772 11384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:34425 every 8 connection(s)
I20260619 12:05:31.445083 11256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:05:31.445922 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 11256
I20260619 12:05:31.446040 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:05:31.446293 814 external_mini_cluster.cc:1428] Setting key 0a9ce6ee876814bc5dea016de4b20515
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
I20260619 12:05:31.457631 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.445959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:44833 (local address 127.0.203.190:42041)
0619 12:05:31.446054 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.446057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.446342 (+ 285us) server_negotiation.cc:407] Connection header received
0619 12:05:31.447095 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.447096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.447109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.447176 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:31.448702 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.449235 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.450672 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.450897 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.450995 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.453880 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:31.453882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:31.453882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:31.453891 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:31.455501 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.456187 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.456190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.456190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.456280 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.456714 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.456715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.456715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.456872 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:31.456957 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.457459 (+ 502us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.457554 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:05:31.457849 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.445909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42041 (local address 127.0.203.130:44833)
0619 12:05:31.446118 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.446185 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.446949 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.447287 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.447293 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.448199 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:31.448555 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.448563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.449356 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.449357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.450548 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.450551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.451062 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.451065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.451238 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.451933 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:31.451943 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:31.453752 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:31.455586 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:31.455596 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:31.455599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:31.456034 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:31.456385 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:31.456387 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:31.456387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:31.456569 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:31.456994 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:31.457001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:31.457245 (+ 244us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.457394 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.457581 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260619 12:05:31.458492 11385 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42041
I20260619 12:05:31.458600 11385 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
I20260619 12:05:31.458855 11385 heartbeater.cc:511] Master 127.0.203.190:42041 requested a full tablet report, sending...
I20260619 12:05:31.459517 11060 ts_manager.cc:194] Registered new tserver with Master: 5b34cf9b9d9b4f79938d78db414af376 (127.0.203.130:34425)
I20260619 12:05:31.459949 11060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:44833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:05:31.475672 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42041
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:05:31.564270 11390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:05:31.564523 11390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:05:31.564561 11390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:05:31.565876 11390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:05:31.565932 11390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:05:31.565971 11390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:05:31.567618 11390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:42041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:05:31.567898 11390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:05:31.568127 11390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:05:31.573946 11395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:31.574028 11398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:05:31.574004 11396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:05:31.574700 11390 server_base.cc:1034] running on GCE node
I20260619 12:05:31.574990 11390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:05:31.575271 11390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:05:31.576454 11390 hybrid_clock.cc:648] HybridClock initialized: now 1781870731576419 us; error 65 us; skew 500 ppm
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:05:31.579700 11390 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:05:31.580318 11390 webserver.cc:460] Webserver started at http://127.0.203.131:44709/ using document root <none> and password file <none>
I20260619 12:05:31.580569 11390 fs_manager.cc:359] Metadata directory not provided
I20260619 12:05:31.580624 11390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:05:31.580753 11390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:05:31.581988 11390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "3b0bc806df394afd8fafed44bd5e7a00"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "67b8f0955def001139a1c203c8395f79"
server_key_iv: "a124992985c5b5288a5a4f52208bdcf7"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.582337 11390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "3b0bc806df394afd8fafed44bd5e7a00"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "67b8f0955def001139a1c203c8395f79"
server_key_iv: "a124992985c5b5288a5a4f52208bdcf7"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.583866 11390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:05:31.584851 11405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.585165 11390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:05:31.585278 11390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "3b0bc806df394afd8fafed44bd5e7a00"
format_stamp: "Formatted at 2026-06-19 12:05:31 on dist-test-slave-44vj"
server_key: "67b8f0955def001139a1c203c8395f79"
server_key_iv: "a124992985c5b5288a5a4f52208bdcf7"
server_key_version: "encryptionkey@0"
I20260619 12:05:31.585361 11390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:05:31.595577 11390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:05:31.598055 11390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:05:31.598210 11390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:05:31.598456 11390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:05:31.598778 11390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:05:31.598812 11390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.598843 11390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:05:31.598861 11390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:05:31.606077 11390 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:34711
I20260619 12:05:31.606153 11518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:34711 every 8 connection(s)
I20260619 12:05:31.606611 11390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:05:31.611887 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 11390
I20260619 12:05:31.611997 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:05:31.612195 814 external_mini_cluster.cc:1428] Setting key 4d92dabf77c52a3b138be829e2137553
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:05:31.619442 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.607689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:40553 (local address 127.0.203.190:42041)
0619 12:05:31.607790 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.607793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.608272 (+ 479us) server_negotiation.cc:407] Connection header received
0619 12:05:31.608944 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.608946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.608961 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.609040 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:31.610304 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.610949 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.612179 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.612537 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.612629 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.615294 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:31.615296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:31.615297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:31.615309 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:31.616929 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.617921 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.617923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.617923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.618004 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.618487 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.618488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.618489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.618641 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:31.618715 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.619162 (+ 447us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.619295 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:05:31.619598 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.607653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42041 (local address 127.0.203.131:40553)
0619 12:05:31.607902 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.607960 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.608782 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.609263 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.609268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.609901 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:05:31.610112 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.610116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.611100 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.611101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.612050 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.612055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.612666 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.612667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.612845 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.613445 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:05:31.613456 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:05:31.615156 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:05:31.617137 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:31.617151 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:31.617168 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:31.617672 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:31.618133 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:05:31.618135 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:05:31.618135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:05:31.618357 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:05:31.618734 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:05:31.618741 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:05:31.618994 (+ 253us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.619112 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.619286 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":118,"threads_started":1}
I20260619 12:05:31.620613 11519 heartbeater.cc:347] Connected to a master server at 127.0.203.190:42041
I20260619 12:05:31.620857 11519 heartbeater.cc:464] Registering TS with master...
I20260619 12:05:31.621126 11519 heartbeater.cc:511] Master 127.0.203.190:42041 requested a full tablet report, sending...
I20260619 12:05:31.621840 11060 ts_manager.cc:194] Registered new tserver with Master: 3b0bc806df394afd8fafed44bd5e7a00 (127.0.203.131:34711)
I20260619 12:05:31.622247 11060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:40553
I20260619 12:05:31.624816 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:05:31.633734 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.626658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.0.203.190:42041)
0619 12:05:31.626870 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.626873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.627011 (+ 138us) server_negotiation.cc:407] Connection header received
0619 12:05:31.627072 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.627074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.627085 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.627163 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:05:31.627807 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.628355 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.629143 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.629331 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.629384 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.630705 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:05:31.630707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:05:31.630708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:05:31.630716 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:05:31.632354 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.632766 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.632767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.632768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.632833 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:05:31.633022 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:05:31.633023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:05:31.633024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:05:31.633282 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:05:31.633385 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.633566 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.633643 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260619 12:05:31.635704 11060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:05:31.637467 11060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:05:31.645234 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.641240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.0.203.130:34425)
0619 12:05:31.641560 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.641565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.641576 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:31.641616 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.641619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.641635 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.641708 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:31.641864 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.642619 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.644060 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.644763 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.644842 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.644906 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.645082 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.645124 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":96,"threads_started":1}
I20260619 12:05:31.645419 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.641007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.0.203.129:45457)
0619 12:05:31.641237 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.641244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.641257 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:05:31.641308 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.641310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.641329 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.641422 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:31.641718 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.642482 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.644327 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.644868 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.644939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.645004 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.645275 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.645328 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":106,"threads_started":1}
I20260619 12:05:31.645591 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.640735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:45457 (local address 127.0.0.1:36972)
0619 12:05:31.640961 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.641012 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.641084 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.641515 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.641519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.641523 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:31.641628 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.641631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.642579 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.642580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.644250 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.644253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.645036 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.645037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.645093 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.645096 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.645134 (+ 38us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.645174 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":48,"thread_start_us":84,"threads_started":1}
I20260619 12:05:31.645255 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.641004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34425 (local address 127.0.0.1:40796)
0619 12:05:31.641384 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.641392 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.641429 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.641718 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.641720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.641723 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:31.641801 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.641803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.642708 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.642709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.643988 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.643991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.644862 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.644864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.644957 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.644974 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.645055 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.645127 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":85,"thread_start_us":89,"threads_started":1}
I20260619 12:05:31.646461 11185 tablet_service.cc:1467] Processing CreateTablet for tablet 27225114a72c46c6b869a7af078f04c4 (DEFAULT_TABLE table=test-table [id=a174df1c76864df598c494d4ee83d376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:31.646590 11319 tablet_service.cc:1467] Processing CreateTablet for tablet 27225114a72c46c6b869a7af078f04c4 (DEFAULT_TABLE table=test-table [id=a174df1c76864df598c494d4ee83d376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:31.646917 11319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27225114a72c46c6b869a7af078f04c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:31.646917 11185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27225114a72c46c6b869a7af078f04c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:31.650452 11537 tablet_bootstrap.cc:492] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c: Bootstrap starting.
I20260619 12:05:31.651232 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.640735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:34711 (local address 127.0.0.1:42164)
0619 12:05:31.641242 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.641251 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.641298 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.642874 (+ 1576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.642876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.642878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:31.642926 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.642928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.644246 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.644247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.645285 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.645289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.650883 (+ 5594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.650884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.650994 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.651002 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.651074 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.651139 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":6,"spinlock_wait_cycles":768,"thread_start_us":109,"threads_started":1}
I20260619 12:05:31.651278 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.640888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.0.203.131:34711)
0619 12:05:31.642663 (+ 1775us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.642667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.642707 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:05:31.642749 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.642751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.642769 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.642859 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:31.643030 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.644136 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.645422 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.650752 (+ 5330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.650848 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.650894 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.651146 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.651198 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1730,"thread_start_us":154,"threads_started":1}
I20260619 12:05:31.651486 11537 tablet_bootstrap.cc:654] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:31.652096 11537 log.cc:824] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:31.652212 11453 tablet_service.cc:1467] Processing CreateTablet for tablet 27225114a72c46c6b869a7af078f04c4 (DEFAULT_TABLE table=test-table [id=a174df1c76864df598c494d4ee83d376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:05:31.652637 11453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27225114a72c46c6b869a7af078f04c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:05:31.652666 11536 tablet_bootstrap.cc:492] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Bootstrap starting.
I20260619 12:05:31.653154 11537 tablet_bootstrap.cc:492] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c: No bootstrap required, opened a new log
I20260619 12:05:31.653244 11537 ts_tablet_manager.cc:1397] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 12:05:31.653388 11536 tablet_bootstrap.cc:654] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:31.653980 11536 log.cc:824] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:31.654989 11536 tablet_bootstrap.cc:492] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: No bootstrap required, opened a new log
I20260619 12:05:31.655068 11536 ts_tablet_manager.cc:1397] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:05:31.655017 11537 raft_consensus.cc:348] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.655229 11537 raft_consensus.cc:374] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:31.655274 11537 raft_consensus.cc:729] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7306931add8945a2874c7e4479cac38c, State: Initialized, Role: FOLLOWER
I20260619 12:05:31.655428 11537 consensus_queue.cc:260] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.655766 11251 heartbeater.cc:503] Master 127.0.203.190:42041 was elected leader, sending a full tablet report...
I20260619 12:05:31.656028 11537 ts_tablet_manager.cc:1428] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:05:31.656476 11541 tablet_bootstrap.cc:492] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Bootstrap starting.
I20260619 12:05:31.657272 11541 tablet_bootstrap.cc:654] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Neither blocks nor log segments found. Creating new log.
I20260619 12:05:31.657569 11536 raft_consensus.cc:348] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.657739 11536 raft_consensus.cc:374] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:31.657761 11536 raft_consensus.cc:729] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b34cf9b9d9b4f79938d78db414af376, State: Initialized, Role: FOLLOWER
I20260619 12:05:31.657765 11541 log.cc:824] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Log is configured to *not* fsync() on all Append() calls
I20260619 12:05:31.657864 11536 consensus_queue.cc:260] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.658170 11536 ts_tablet_manager.cc:1428] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:05:31.658357 11385 heartbeater.cc:503] Master 127.0.203.190:42041 was elected leader, sending a full tablet report...
I20260619 12:05:31.658694 11541 tablet_bootstrap.cc:492] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: No bootstrap required, opened a new log
I20260619 12:05:31.658769 11541 ts_tablet_manager.cc:1397] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:05:31.660532 11541 raft_consensus.cc:348] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.660698 11541 raft_consensus.cc:374] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:05:31.660742 11541 raft_consensus.cc:729] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0bc806df394afd8fafed44bd5e7a00, State: Initialized, Role: FOLLOWER
I20260619 12:05:31.660856 11541 consensus_queue.cc:260] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.661175 11541 ts_tablet_manager.cc:1428] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:05:31.661310 11519 heartbeater.cc:503] Master 127.0.203.190:42041 was elected leader, sending a full tablet report...
W20260619 12:05:31.695789 11386 tablet.cc:2369] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:05:31.730916 11540 raft_consensus.cc:482] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:05:31.731096 11540 raft_consensus.cc:504] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.731650 11540 leader_election.cc:288] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b0bc806df394afd8fafed44bd5e7a00 (127.0.203.131:34711), 5b34cf9b9d9b4f79938d78db414af376 (127.0.203.130:34425)
I20260619 12:05:31.734871 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.732148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:41901 (local address 127.0.203.130:34425)
0619 12:05:31.732222 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.732225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.732326 (+ 101us) server_negotiation.cc:407] Connection header received
0619 12:05:31.732389 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.732390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.732400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.732450 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:31.732614 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.733128 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.734081 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.734485 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.734564 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.734576 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.734749 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.734792 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:05:31.734938 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.731905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:34711 (local address 127.0.203.129:58943)
0619 12:05:31.732019 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.732036 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.732134 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.732292 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.732293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.732295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:31.732391 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.732394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.733048 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.733049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.733984 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.733988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.734638 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.734639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.734724 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.734729 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.734790 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.734844 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":61}
I20260619 12:05:31.735112 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.731997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34425 (local address 127.0.203.129:41901)
0619 12:05:31.732234 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.732245 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:05:31.732327 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:05:31.732458 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:05:31.732459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:05:31.732461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:05:31.732555 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.732557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.733203 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.733204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.733996 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:05:31.733998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.734587 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.734588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:05:31.734672 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.734676 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:05:31.734740 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:05:31.734800 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":143,"threads_started":1}
I20260619 12:05:31.735184 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.732115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:58943 (local address 127.0.203.131:34711)
0619 12:05:31.732162 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.732164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.732187 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:05:31.732216 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.732217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.732226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.732282 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:05:31.732469 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.732974 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.734085 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.734527 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.734598 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.734608 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.735062 (+ 454us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.735127 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:05:31.735527 11473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27225114a72c46c6b869a7af078f04c4" candidate_uuid: "7306931add8945a2874c7e4479cac38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0bc806df394afd8fafed44bd5e7a00" is_pre_election: true
I20260619 12:05:31.735539 11339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27225114a72c46c6b869a7af078f04c4" candidate_uuid: "7306931add8945a2874c7e4479cac38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b34cf9b9d9b4f79938d78db414af376" is_pre_election: true
I20260619 12:05:31.735702 11473 raft_consensus.cc:2454] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7306931add8945a2874c7e4479cac38c in term 0.
I20260619 12:05:31.735720 11339 raft_consensus.cc:2454] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7306931add8945a2874c7e4479cac38c in term 0.
I20260619 12:05:31.735944 11140 leader_election.cc:302] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b0bc806df394afd8fafed44bd5e7a00, 7306931add8945a2874c7e4479cac38c; no voters:
I20260619 12:05:31.736107 11540 raft_consensus.cc:2781] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:05:31.736195 11540 raft_consensus.cc:482] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:05:31.736213 11540 raft_consensus.cc:3037] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:31.737130 11540 raft_consensus.cc:504] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.737272 11540 leader_election.cc:288] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [CANDIDATE]: Term 1 election: Requested vote from peers 3b0bc806df394afd8fafed44bd5e7a00 (127.0.203.131:34711), 5b34cf9b9d9b4f79938d78db414af376 (127.0.203.130:34425)
I20260619 12:05:31.737569 11473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27225114a72c46c6b869a7af078f04c4" candidate_uuid: "7306931add8945a2874c7e4479cac38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0bc806df394afd8fafed44bd5e7a00"
I20260619 12:05:31.737708 11473 raft_consensus.cc:3037] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:31.737658 11339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27225114a72c46c6b869a7af078f04c4" candidate_uuid: "7306931add8945a2874c7e4479cac38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b34cf9b9d9b4f79938d78db414af376"
I20260619 12:05:31.737808 11339 raft_consensus.cc:3037] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:05:31.738859 11339 raft_consensus.cc:2454] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7306931add8945a2874c7e4479cac38c in term 1.
I20260619 12:05:31.738859 11473 raft_consensus.cc:2454] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7306931add8945a2874c7e4479cac38c in term 1.
I20260619 12:05:31.739109 11140 leader_election.cc:302] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b0bc806df394afd8fafed44bd5e7a00, 7306931add8945a2874c7e4479cac38c; no voters:
I20260619 12:05:31.739249 11540 raft_consensus.cc:2781] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:05:31.739498 11540 raft_consensus.cc:686] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [term 1 LEADER]: Becoming Leader. State: Replica: 7306931add8945a2874c7e4479cac38c, State: Running, Role: LEADER
I20260619 12:05:31.739612 11540 consensus_queue.cc:237] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } }
I20260619 12:05:31.740744 11060 catalog_manager.cc:5495] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7306931add8945a2874c7e4479cac38c (127.0.203.129). New cstate: current_term: 1 leader_uuid: "7306931add8945a2874c7e4479cac38c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7306931add8945a2874c7e4479cac38c" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 45457 } health_report { overall_health: HEALTHY } } }
I20260619 12:05:31.764271 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.761162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36980 (local address 127.0.203.129:45457)
0619 12:05:31.761228 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:05:31.761230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:05:31.761241 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:05:31.761280 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:05:31.761281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:05:31.761299 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:05:31.761369 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:05:31.761629 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.762166 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.763043 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:05:31.763302 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:05:31.763415 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:05:31.763781 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:05:31.763913 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:05:31.764002 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:05:31.764137 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 12:05:31.764167 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:05:31.766746 11473 raft_consensus.cc:1261] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 1 FOLLOWER]: Refusing update from remote peer 7306931add8945a2874c7e4479cac38c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:31.766793 11339 raft_consensus.cc:1261] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 1 FOLLOWER]: Refusing update from remote peer 7306931add8945a2874c7e4479cac38c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:05:31.767293 11540 consensus_queue.cc:1035] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b0bc806df394afd8fafed44bd5e7a00" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 34711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:31.767539 11546 consensus_queue.cc:1035] T 27225114a72c46c6b869a7af078f04c4 P 7306931add8945a2874c7e4479cac38c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b34cf9b9d9b4f79938d78db414af376" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:05:31.770479 11549 mvcc.cc:205] Tried to move back new op lower bound from 7298542517313306624 to 7298542517206159360. Current Snapshot: MvccSnapshot[applied={T|T < 7298542517313306624}]
I20260619 12:05:31.770882 11550 mvcc.cc:205] Tried to move back new op lower bound from 7298542517313306624 to 7298542517206159360. Current Snapshot: MvccSnapshot[applied={T|T < 7298542517313306624}]
I20260619 12:05:31.771291 11551 mvcc.cc:205] Tried to move back new op lower bound from 7298542517313306624 to 7298542517206159360. Current Snapshot: MvccSnapshot[applied={T|T < 7298542517313306624}]
I20260619 12:05:31.773939 11059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:31.774060 11059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:05:31.774999 11059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 545cc94b73554369a73f18447e7a1da7: Sending DeleteTablet for 3 replicas of tablet 27225114a72c46c6b869a7af078f04c4
I20260619 12:05:31.775426 11185 tablet_service.cc:1514] Processing DeleteTablet for tablet 27225114a72c46c6b869a7af078f04c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:31 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:36972
I20260619 12:05:31.775481 11319 tablet_service.cc:1514] Processing DeleteTablet for tablet 27225114a72c46c6b869a7af078f04c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:31 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:40796
I20260619 12:05:31.775635 11453 tablet_service.cc:1514] Processing DeleteTablet for tablet 27225114a72c46c6b869a7af078f04c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:05:31 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42164
I20260619 12:05:31.775887 11561 tablet_replica.cc:330] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: stopping tablet replica
I20260619 12:05:31.776021 11561 raft_consensus.cc:2229] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:31.776018 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 11122
I20260619 12:05:31.776880 11561 raft_consensus.cc:2258] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:31.777500 11561 ts_tablet_manager.cc:1905] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:31.778012 11560 tablet_replica.cc:330] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: stopping tablet replica
I20260619 12:05:31.778182 11560 raft_consensus.cc:2229] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:05:31.778321 11560 raft_consensus.cc:2258] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:05:31.778936 11560 ts_tablet_manager.cc:1905] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:05:31.779588 11561 ts_tablet_manager.cc:1918] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:31.779691 11561 log.cc:1196] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/27225114a72c46c6b869a7af078f04c4
I20260619 12:05:31.779978 11561 ts_tablet_manager.cc:1939] T 27225114a72c46c6b869a7af078f04c4 P 3b0bc806df394afd8fafed44bd5e7a00: Deleting consensus metadata
I20260619 12:05:31.780596 11046 catalog_manager.cc:4841] TS 3b0bc806df394afd8fafed44bd5e7a00 (127.0.203.131:34711): tablet 27225114a72c46c6b869a7af078f04c4 (table test-table [id=a174df1c76864df598c494d4ee83d376]) successfully deleted
I20260619 12:05:31.780890 11560 ts_tablet_manager.cc:1918] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:05:31.780984 11560 log.cc:1196] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/27225114a72c46c6b869a7af078f04c4
I20260619 12:05:31.781270 11560 ts_tablet_manager.cc:1939] T 27225114a72c46c6b869a7af078f04c4 P 5b34cf9b9d9b4f79938d78db414af376: Deleting consensus metadata
I20260619 12:05:31.781819 11046 catalog_manager.cc:4841] TS 5b34cf9b9d9b4f79938d78db414af376 (127.0.203.130:34425): tablet 27225114a72c46c6b869a7af078f04c4 (table test-table [id=a174df1c76864df598c494d4ee83d376]) successfully deleted
W20260619 12:05:31.782963 11044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:45457 (error 108)
I20260619 12:05:31.783623 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 11256
I20260619 12:05:31.783716 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:05:31.783553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:45457 (local address 127.0.0.1:36994)
0619 12:05:31.783624 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 12:05:31.783675 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:45457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:05:31.785099 11044 catalog_manager.cc:4584] TS 7306931add8945a2874c7e4479cac38c (127.0.203.129:45457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 27225114a72c46c6b869a7af078f04c4: Network error: Client connection negotiation failed: client connection to 127.0.203.129:45457: connect: Connection refused (error 111)
I20260619 12:05:31.788607 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 11390
I20260619 12:05:31.793457 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 11028
2026-06-19T12:05:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1959 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11565](info): set up 2 sockets
Jun 19 12:05:31 dist-test-slave-44vj krb5kdc[11565](info): commencing operation
krb5kdc: starting...
W20260619 12:05:32.764523 814 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:05:32 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:05:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:05:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/rangeradmin_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/rangeradmin_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/rangerlookup_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/rangerlookup_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
I20260619 12:05:32.942802 814 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskgwdYux/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/postgres -l logfile start
2026-06-19 12:05:35.556 UTC [11595] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 12:05:35.556 UTC [11595] LOG: listening on IPv4 address "127.0.203.148", port 33703
2026-06-19 12:05:35.559 UTC [11595] LOG: listening on Unix socket "/tmp/.s.PGSQL.33703"
2026-06-19 12:05:35.589 UTC [11598] LOG: database system was shut down at 2026-06-19 12:05:34 UTC
2026-06-19 12:05:35.593 UTC [11595] LOG: database system is ready to accept connections
I20260619 12:05:36.468612 814 mini_postgres.cc:96] Postgres bound to 33703
2026-06-19 12:05:36.475 UTC [11606] FATAL: database "slave" does not exist
127.0.203.148:33703 - accepting connections
I20260619 12:05:36.476280 814 mini_ranger.cc:161] Starting Ranger...
I20260619 12:05:36.493230 814 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 12:05:37.004282 814 mini_ranger.cc:87] Created ranger Postgres database
I20260619 12:05:37.004369 814 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-admin
2026-06-19 12:05:37,295 [I] DB FLAVOR :POSTGRES
2026-06-19 12:05:37,295 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:05:37,295 [I] Checking connection..
2026-06-19 12:05:37,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 12:05:37,589 [I] Checking connection passed.
2026-06-19 12:05:37,589 [I] --------- Verifying version history table ---------
2026-06-19 12:05:37,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:05:37,870 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:05:37,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:05:38,159 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:05:38,159 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 12:05:38,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 12:05:38.427 UTC [11670] WARNING: there is no transaction in progress
2026-06-19 12:05:38,433 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 12:05:38,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:05:38,703 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 12:05:38,703 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:05:38,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 12:05:38,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 12:05:39,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-44vj.c.gcp-upstream.internal','N') ;"
2026-06-19 12:05:39,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:05:39,787 [I] Table x_portal_user does not exist in database ranger
2026-06-19 12:05:39,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:05:40,082 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 12:05:40,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:05:40,344 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:05:40,623 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 12:05:40,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 12:05:41.757 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.778 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.797 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.817 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.837 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.859 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.920 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.928 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.943 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.955 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.966 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.982 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:41.993 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:42.000 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:42.008 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:42.016 UTC [11778] WARNING: there is no transaction in progress
2026-06-19 12:05:42,781 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 12:05:42,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 12:05:43,037 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 12:05:43,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:05:43,307 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 12:05:43,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:05:43,584 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 12:05:43,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:05:43,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:05:44,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-44vj.c.gcp-upstream.internal';"
2026-06-19 12:05:44,409 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 12:05:44,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 12:05:44,690 [I] DB_PATCHES have already been applied
I20260619 12:05:44.697576 814 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-admin:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskgwdYux/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 12:05:44.697675 814 mini_ranger.cc:193] Using host: 127.0.203.148
I20260619 12:05:44.701400 814 mini_ranger.cc:227] Ranger admin URL: http://127.0.203.148:59227
Jun 19, 2026 12:05:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:05:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:05:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 12:05:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 12:05:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 12:05:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 12:05:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 12:05:45.717743 814 mini_ranger.cc:160] Time spent starting Ranger: real 9.241s user 0.000s sys 0.010s
Jun 19, 2026 12:06:08 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 12:06:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 19, 2026 12:06:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 12:06:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 12:06:09 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 12:06:10 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 12:06:10.707302 814 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/rangerkms_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/rangerkms_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.203.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.203.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 12:06:10 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870770, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 12:06:10.798054 814 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 12:06:10.810354 814 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 12:06:11.751852 814 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 12:06:11.751950 814 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms
I20260619 12:06:11.751963 814 mini_ranger_kms.cc:227] Using postgres at 127.0.203.148:33703
2026-06-19 12:06:11,960 [I] DB FLAVOR :POSTGRES
2026-06-19 12:06:11,960 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:06:11,960 [I] Checking connection
2026-06-19 12:06:11,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 12:06:12,228 [I] connection success
2026-06-19 12:06:12,229 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 12:06:12,229 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 12:06:12,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 12:06:12,499 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 12:06:12,500 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:06:12,500 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 12:06:12,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:33703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 12:06:12,820 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 12:06:13.408527 814 mini_ranger_kms.cc:310] Created kms service
I20260619 12:06:13.618644 814 mini_ranger_kms.cc:326] Created kudu user
I20260619 12:06:13.674984 814 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 12:06:13 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870770, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:06:14.111297 814 mini_ranger_kms.cc:384] Added ranger policy
I20260619 12:06:14.111506 814 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms:/tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 12:06:14.111527 814 mini_ranger_kms.cc:240] Using host: 127.0.203.148
I20260619 12:06:14.116551 814 mini_ranger_kms.cc:276] Ranger KMS PID: 11959
I20260619 12:06:14.116683 814 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.203.148:37811
Jun 19, 2026 12:06:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:06:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:06:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 12:06:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 12:06:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 12:06:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 12:06:16.159278 814 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.361s user 0.002s sys 0.190s
I20260619 12:06:16.159443 814 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 12:06:16.159540 814 mini_ranger_kms.cc:401] 127.0.203.148:37811/kms/v1/keys
Jun 19 12:06:16 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781870776, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.203.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 12:06:16 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870776, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.203.148@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
Jun 19 12:06:17 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870776, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.203.148@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
Jun 19 12:06:19 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:06:19.970479 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36729
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:36729
--ranger_config_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:37811/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:06:20.073349 12009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:06:20.073689 12009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:06:20.073729 12009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:06:20.075469 12009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:06:20.075546 12009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:06:20.075567 12009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:06:20.075582 12009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:06:20.075598 12009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:06:20.077931 12009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.203.148:37811/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:36729
--ranger_config_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:06:20.078295 12009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:06:20.078567 12009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:06:20.084779 12015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:06:20.085096 12009 server_base.cc:1034] running on GCE node
W20260619 12:06:20.084779 12014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:06:20.084828 12017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:06:20.085777 12009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:06:20.086241 12009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:06:20.087446 12009 hybrid_clock.cc:648] HybridClock initialized: now 1781870780087412 us; error 62 us; skew 500 ppm
Jun 19 12:06:20 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:06:20.091034 12009 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:06:20.091735 12009 webserver.cc:460] Webserver started at http://127.0.203.190:45819/ using document root <none> and password file <none>
I20260619 12:06:20.092059 12009 fs_manager.cc:359] Metadata directory not provided
I20260619 12:06:20.092113 12009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:06:20.092264 12009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:06:20 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:06:20.148813 12009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "cf448d7dbb22445db8599ec9316c9c46"
format_stamp: "Formatted at 2026-06-19 12:06:20 on dist-test-slave-44vj"
server_key: "fae4f034705923ee3bb3dd6d03ac2073"
server_key_iv: "7705d475a71b2b172d7e4572a92a9063"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:20.149261 12009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "cf448d7dbb22445db8599ec9316c9c46"
format_stamp: "Formatted at 2026-06-19 12:06:20 on dist-test-slave-44vj"
server_key: "fae4f034705923ee3bb3dd6d03ac2073"
server_key_iv: "7705d475a71b2b172d7e4572a92a9063"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:20.150769 12009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:06:20.165407 12024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:20.165678 12009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:06:20.165774 12009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "cf448d7dbb22445db8599ec9316c9c46"
format_stamp: "Formatted at 2026-06-19 12:06:20 on dist-test-slave-44vj"
server_key: "fae4f034705923ee3bb3dd6d03ac2073"
server_key_iv: "7705d475a71b2b172d7e4572a92a9063"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:20.165894 12009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:06:20.188604 12009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:06:20.190955 12009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:06:20.191135 12009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:06:20.196980 12009 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:36729
I20260619 12:06:20.197012 12076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:36729 every 8 connection(s)
I20260619 12:06:20.197468 12009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:06:20.198395 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 12009
I20260619 12:06:20.198436 12077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:06:20.198494 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:06:20.198705 814 external_mini_cluster.cc:1428] Setting key d0ceda1e5a7309c41199f74729860a59
I20260619 12:06:20.202852 12077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46: Bootstrap starting.
I20260619 12:06:20.204237 12077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46: Neither blocks nor log segments found. Creating new log.
I20260619 12:06:20.204592 12077 log.cc:824] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46: Log is configured to *not* fsync() on all Append() calls
I20260619 12:06:20.205619 12077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46: No bootstrap required, opened a new log
Jun 19 12:06:20 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:06:20.208508 12077 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf448d7dbb22445db8599ec9316c9c46" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36729 } }
I20260619 12:06:20.208933 12077 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:06:20.209110 12077 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf448d7dbb22445db8599ec9316c9c46, State: Initialized, Role: FOLLOWER
I20260619 12:06:20.209359 12080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:20.199311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.0.203.190:36729)
0619 12:06:20.199517 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:20.199526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:20.199552 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:06:20.200449 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:20.200458 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:20.200513 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:20.200660 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:06:20.201315 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:20.202289 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:20.203077 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:20.203586 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:20.203689 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:20.206218 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:06:20.206224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:06:20.206224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:06:20.206236 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:06:20.207850 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:20.208164 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:20.208168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:20.208168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:20.208249 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:20.208416 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:20.208417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:20.208417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:20.208696 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:06:20.208851 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:20.208957 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 12:06:20.209082 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260619 12:06:20.209385 12077 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf448d7dbb22445db8599ec9316c9c46" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36729 } }
I20260619 12:06:20.209491 12077 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:06:20.209606 12077 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:06:20.209877 12077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:06:20.210822 12077 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf448d7dbb22445db8599ec9316c9c46" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36729 } }
I20260619 12:06:20.210984 12077 leader_election.cc:302] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf448d7dbb22445db8599ec9316c9c46; no voters:
I20260619 12:06:20.211217 12077 leader_election.cc:288] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:06:20.211272 12082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:06:20.211534 12082 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [term 1 LEADER]: Becoming Leader. State: Replica: cf448d7dbb22445db8599ec9316c9c46, State: Running, Role: LEADER
I20260619 12:06:20.211551 12077 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:06:20.211658 12082 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf448d7dbb22445db8599ec9316c9c46" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36729 } }
I20260619 12:06:20.212078 12083 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf448d7dbb22445db8599ec9316c9c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf448d7dbb22445db8599ec9316c9c46" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36729 } } }
I20260619 12:06:20.212069 12082 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf448d7dbb22445db8599ec9316c9c46. Latest consensus state: current_term: 1 leader_uuid: "cf448d7dbb22445db8599ec9316c9c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf448d7dbb22445db8599ec9316c9c46" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36729 } } }
I20260619 12:06:20.212236 12083 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [sys.catalog]: This master's current role is: LEADER
I20260619 12:06:20.212244 12082 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46 [sys.catalog]: This master's current role is: LEADER
I20260619 12:06:20.212536 12091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:06:20.213161 12091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:06:20.213515 12077 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:06:20.219900 12091 catalog_manager.cc:1347] Generated new cluster ID: 2adb9305b7e24a8183495b6b9f0497fa
I20260619 12:06:20.219990 12091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:06:20.229068 12091 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:06:20.230038 12091 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:06:20.239677 12091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46: Generated new TSK 0
I20260619 12:06:20.240041 12091 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:06:21.143045 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:37811/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36729
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:06:21.252014 12122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:06:21.252249 12122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:06:21.252285 12122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:06:21.254016 12122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:06:21.254091 12122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:06:21.254136 12122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:06:21.256475 12122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.203.148:37811/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:06:21.256772 12122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:06:21.257119 12122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:06:21.263665 12128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:06:21.264259 12122 server_base.cc:1034] running on GCE node
W20260619 12:06:21.264377 12127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:06:21.264402 12130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:06:21.264849 12122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:06:21.265232 12122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:06:21.266428 12122 hybrid_clock.cc:648] HybridClock initialized: now 1781870781266410 us; error 43 us; skew 500 ppm
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:06:21.270229 12122 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:06:21.270884 12122 webserver.cc:460] Webserver started at http://127.0.203.129:44453/ using document root <none> and password file <none>
I20260619 12:06:21.271164 12122 fs_manager.cc:359] Metadata directory not provided
I20260619 12:06:21.271245 12122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:06:21.271486 12122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:06:21.288738 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "3c03c1349da5476e95cd613a713cf81e"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "1b1599bcdb12124c9670630977044460"
server_key_iv: "d154c76b325d814a2c7c7d096893eb80"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.289878 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "3c03c1349da5476e95cd613a713cf81e"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "1b1599bcdb12124c9670630977044460"
server_key_iv: "d154c76b325d814a2c7c7d096893eb80"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.292392 12122 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260619 12:06:21.306574 12137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.306769 12122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:06:21.306857 12122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "3c03c1349da5476e95cd613a713cf81e"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "1b1599bcdb12124c9670630977044460"
server_key_iv: "d154c76b325d814a2c7c7d096893eb80"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.306955 12122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:06:21.336153 12122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:06:21.338630 12122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:06:21.338858 12122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:06:21.339183 12122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:06:21.339656 12122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:06:21.339745 12122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.339843 12122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:06:21.339915 12122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.349162 12122 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:35215
I20260619 12:06:21.349411 12252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:35215 every 8 connection(s)
I20260619 12:06:21.349867 12122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:06:21.351554 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 12122
I20260619 12:06:21.351657 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:06:21.351878 814 external_mini_cluster.cc:1428] Setting key 313fb396f1383866bc5a49235d2e6e4a
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:06:21.365509 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.352073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:36729 (local address 127.0.203.129:37113)
0619 12:06:21.352584 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:21.352637 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:21.353366 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:21.353862 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:21.353867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:21.354176 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:06:21.354520 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.354526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.355752 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.355755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.357174 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.357179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.357857 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.357858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.358070 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.358824 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:06:21.358838 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:06:21.361157 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:06:21.363528 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:06:21.363543 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:06:21.363546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:06:21.363931 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:06:21.364202 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:06:21.364203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:06:21.364203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:06:21.364373 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:06:21.364721 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:06:21.364729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:06:21.364923 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 12:06:21.365029 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:06:21.365237 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":125,"threads_started":1}
I20260619 12:06:21.366487 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.352083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:37113 (local address 127.0.203.190:36729)
0619 12:06:21.353587 (+ 1504us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:21.353589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:21.353597 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:06:21.353625 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:21.353625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:21.353641 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:21.353704 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:06:21.354656 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.355556 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.357317 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.357695 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.357789 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.361348 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:06:21.361350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:06:21.361351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:06:21.361362 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:06:21.363402 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.364033 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.364034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.364034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.364106 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.364475 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.364476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.364477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.364637 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:06:21.366223 (+ 1586us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:21.366274 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 12:06:21.366380 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1474,"thread_start_us":89,"threads_started":1}
I20260619 12:06:21.367152 12253 heartbeater.cc:347] Connected to a master server at 127.0.203.190:36729
I20260619 12:06:21.367280 12253 heartbeater.cc:464] Registering TS with master...
I20260619 12:06:21.367605 12253 heartbeater.cc:511] Master 127.0.203.190:36729 requested a full tablet report, sending...
I20260619 12:06:21.368259 12041 ts_manager.cc:194] Registered new tserver with Master: 3c03c1349da5476e95cd613a713cf81e (127.0.203.129:35215)
I20260619 12:06:21.369232 12041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:37113
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:06:21.395551 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:37811/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36729
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:06:21.503799 12259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:06:21.504046 12259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:06:21.504081 12259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:06:21.505395 12259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:06:21.505448 12259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:06:21.505487 12259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:06:21.507098 12259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.203.148:37811/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:06:21.507395 12259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:06:21.507650 12259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:06:21.512888 12269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:06:21.513177 12267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:06:21.513444 12259 server_base.cc:1034] running on GCE node
W20260619 12:06:21.513401 12266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:06:21.513833 12259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:06:21.514132 12259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:06:21.515353 12259 hybrid_clock.cc:648] HybridClock initialized: now 1781870781515316 us; error 59 us; skew 500 ppm
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:06:21.519249 12259 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:06:21.520361 12259 webserver.cc:460] Webserver started at http://127.0.203.130:40803/ using document root <none> and password file <none>
I20260619 12:06:21.520733 12259 fs_manager.cc:359] Metadata directory not provided
I20260619 12:06:21.520859 12259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:06:21.521044 12259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:06:21.545836 12259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "332e2ecd0b394ababdb83b423dd0a667"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "39ae0853052b74b5fa542219b8996622"
server_key_iv: "0d0af320162553602bb0d399118c4c3a"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.546514 12259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "332e2ecd0b394ababdb83b423dd0a667"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "39ae0853052b74b5fa542219b8996622"
server_key_iv: "0d0af320162553602bb0d399118c4c3a"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.551992 12259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260619 12:06:21.580682 12276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.580953 12259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:06:21.581053 12259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "332e2ecd0b394ababdb83b423dd0a667"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "39ae0853052b74b5fa542219b8996622"
server_key_iv: "0d0af320162553602bb0d399118c4c3a"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.581161 12259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:06:21.609721 12259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:06:21.612609 12259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:06:21.612839 12259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:06:21.613099 12259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:06:21.613538 12259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:06:21.613627 12259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.613711 12259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:06:21.613766 12259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.622875 12259 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:46037
I20260619 12:06:21.623140 12389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:46037 every 8 connection(s)
I20260619 12:06:21.623533 12259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:06:21.623888 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 12259
I20260619 12:06:21.623992 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:06:21.624213 814 external_mini_cluster.cc:1428] Setting key 138422792f015e9fd07e083392b34c08
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
I20260619 12:06:21.642709 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.627474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:52141 (local address 127.0.203.190:36729)
0619 12:06:21.627521 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:21.627523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:21.631435 (+ 3912us) server_negotiation.cc:407] Connection header received
0619 12:06:21.632275 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:21.632278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:21.632299 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:21.632393 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:06:21.633347 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.634070 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.635640 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.635933 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.635983 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.638978 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:06:21.638980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:06:21.638981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:06:21.638989 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:06:21.640941 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.641548 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.641550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.641550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.641611 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.641913 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.641914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.641914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.642058 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:06:21.642098 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:21.642495 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 12:06:21.642593 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:06:21.643280 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.627118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:36729 (local address 127.0.203.130:52141)
0619 12:06:21.631145 (+ 4027us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:21.631220 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:21.632112 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:21.632595 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:21.632601 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:21.632902 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:06:21.633212 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.633220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.634208 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.634209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.635529 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.635533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.636087 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.636088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.636240 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.636992 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:06:21.637003 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:06:21.638844 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:06:21.641073 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:06:21.641083 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:06:21.641087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:06:21.641437 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:06:21.641692 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:06:21.641693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:06:21.641693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:06:21.641843 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:06:21.642152 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:06:21.642158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:06:21.642339 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 12:06:21.642857 (+ 518us) client_negotiation.cc:240] Negotiation successful
0619 12:06:21.643038 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3954,"thread_start_us":102,"threads_started":1}
I20260619 12:06:21.643913 12390 heartbeater.cc:347] Connected to a master server at 127.0.203.190:36729
I20260619 12:06:21.644032 12390 heartbeater.cc:464] Registering TS with master...
I20260619 12:06:21.644264 12390 heartbeater.cc:511] Master 127.0.203.190:36729 requested a full tablet report, sending...
I20260619 12:06:21.644639 12041 ts_manager.cc:194] Registered new tserver with Master: 332e2ecd0b394ababdb83b423dd0a667 (127.0.203.130:46037)
I20260619 12:06:21.645144 12041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:52141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:06:21.661751 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:37811/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36729
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:06:21.792917 12396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:06:21.793259 12396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:06:21.793308 12396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:06:21.795542 12396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:06:21.795643 12396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:06:21.795692 12396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:06:21.798508 12396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.203.148:37811/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:06:21.798813 12396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:06:21.799108 12396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:06:21.806461 12402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:06:21.806823 12401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:06:21.807076 12404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:06:21.808557 12396 server_base.cc:1034] running on GCE node
I20260619 12:06:21.808908 12396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:06:21.809280 12396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:06:21.816664 12396 hybrid_clock.cc:648] HybridClock initialized: now 1781870781816629 us; error 57 us; skew 500 ppm
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:06:21.820938 12396 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:06:21.821622 12396 webserver.cc:460] Webserver started at http://127.0.203.131:42509/ using document root <none> and password file <none>
I20260619 12:06:21.821895 12396 fs_manager.cc:359] Metadata directory not provided
I20260619 12:06:21.821970 12396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:06:21.822117 12396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:06:21.845603 12396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "9dee6007366a4d12afd84e3a1ee9da7f"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "b8ca9934e38a6ac25a101bf9d25fb9ca"
server_key_iv: "4706340e5aa46b0fe7f151846818f09e"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.846156 12396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "9dee6007366a4d12afd84e3a1ee9da7f"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "b8ca9934e38a6ac25a101bf9d25fb9ca"
server_key_iv: "4706340e5aa46b0fe7f151846818f09e"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.856603 12396 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260619 12:06:21.873045 12411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.873435 12396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260619 12:06:21.873548 12396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "9dee6007366a4d12afd84e3a1ee9da7f"
format_stamp: "Formatted at 2026-06-19 12:06:21 on dist-test-slave-44vj"
server_key: "b8ca9934e38a6ac25a101bf9d25fb9ca"
server_key_iv: "4706340e5aa46b0fe7f151846818f09e"
server_key_version: "kuduclusterkey@0"
I20260619 12:06:21.873674 12396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:06:21.886137 12396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:06:21.888617 12396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:06:21.888769 12396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:06:21.889019 12396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:06:21.889433 12396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:06:21.889483 12396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.889523 12396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:06:21.889539 12396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:06:21.897910 12396 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:44509
I20260619 12:06:21.898295 12524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:44509 every 8 connection(s)
I20260619 12:06:21.898408 12396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:06:21.903167 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 12396
I20260619 12:06:21.903293 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:06:21.903659 814 external_mini_cluster.cc:1428] Setting key 92e0b31ec9a040e8703a31d3f87593e0
Jun 19 12:06:21 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:06:21.916610 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.899422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:50685 (local address 127.0.203.190:36729)
0619 12:06:21.899495 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:21.899498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:21.899816 (+ 318us) server_negotiation.cc:407] Connection header received
0619 12:06:21.900653 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:21.900656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:21.900674 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:21.900800 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:06:21.901624 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.902388 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.903978 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.904294 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.904337 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.912433 (+ 8096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:06:21.912435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:06:21.912436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:06:21.912446 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:06:21.914199 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.915456 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.915457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.915457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.915524 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.915831 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.915832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.915832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.915981 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:06:21.916032 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:21.916421 (+ 389us) server_negotiation.cc:299] Negotiation successful
0619 12:06:21.916509 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:06:21.917448 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.899418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:36729 (local address 127.0.203.131:50685)
0619 12:06:21.899639 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:21.899695 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:21.900481 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:21.900899 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:21.900904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:21.901178 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:06:21.901504 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.901512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.902517 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.902519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.903856 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.903861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.904454 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.904455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.909282 (+ 4827us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.910002 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:06:21.910013 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:06:21.912274 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:06:21.914335 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:06:21.914345 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:06:21.914348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:06:21.915333 (+ 985us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:06:21.915593 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:06:21.915594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:06:21.915594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:06:21.915753 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:06:21.916068 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:06:21.916075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:06:21.916264 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 12:06:21.917010 (+ 746us) client_negotiation.cc:240] Negotiation successful
0619 12:06:21.917199 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":122,"threads_started":1}
I20260619 12:06:21.918100 12525 heartbeater.cc:347] Connected to a master server at 127.0.203.190:36729
I20260619 12:06:21.918193 12525 heartbeater.cc:464] Registering TS with master...
I20260619 12:06:21.918387 12525 heartbeater.cc:511] Master 127.0.203.190:36729 requested a full tablet report, sending...
I20260619 12:06:21.918793 12041 ts_manager.cc:194] Registered new tserver with Master: 9dee6007366a4d12afd84e3a1ee9da7f (127.0.203.131:44509)
I20260619 12:06:21.919261 12041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:50685
I20260619 12:06:21.926262 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:06:21.936088 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.928328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.0.203.190:36729)
0619 12:06:21.928382 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:21.928384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:21.928711 (+ 327us) server_negotiation.cc:407] Connection header received
0619 12:06:21.928804 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:21.928805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:21.928816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:21.928844 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:06:21.929682 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.930387 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.931443 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.931701 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.931740 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.933278 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:06:21.933280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:06:21.933281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:06:21.933288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:06:21.934845 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.935210 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.935211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.935212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.935267 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:06:21.935495 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:06:21.935496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:06:21.935497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:06:21.935631 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:06:21.935708 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:21.935905 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 12:06:21.935983 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:06:21.939316 12041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:06:21.941974 12041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:06:21.953373 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.945600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44509 (local address 127.0.0.1:34608)
0619 12:06:21.947589 (+ 1989us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:21.947659 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:21.947750 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:21.948278 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:21.948282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:21.948285 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:06:21.948377 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.948379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.950070 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.950071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.951445 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.951449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.953020 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.953021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.953119 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.953135 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 12:06:21.953204 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 12:06:21.953268 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"thread_start_us":89,"threads_started":1}
I20260619 12:06:21.953768 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.946791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.0.203.131:44509)
0619 12:06:21.948033 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:21.948039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:21.948050 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:06:21.948090 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:21.948092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:21.948110 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:21.948180 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:06:21.948461 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.949970 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.951989 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.952797 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.952861 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.952945 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:21.953626 (+ 681us) server_negotiation.cc:299] Negotiation successful
0619 12:06:21.953673 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":100,"threads_started":1}
I20260619 12:06:21.954494 12459 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed48520e57d41179b179373c91fa10a (DEFAULT_TABLE table=test-table [id=c8d54f41af6c4d6d9da26c65553eec0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:06:21.954914 12459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed48520e57d41179b179373c91fa10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:06:21.956220 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.946511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.0.203.129:35215)
0619 12:06:21.946746 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:21.946754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:21.951647 (+ 4893us) server_negotiation.cc:407] Connection header received
0619 12:06:21.951793 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:21.951797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:21.951821 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:21.951895 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:06:21.952198 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.953380 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.954818 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.955692 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.955747 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.955815 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:21.956061 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 12:06:21.956119 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":129,"threads_started":1}
I20260619 12:06:21.956550 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.946034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35215 (local address 127.0.0.1:47340)
0619 12:06:21.951500 (+ 5466us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:21.951521 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:21.951692 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:21.952040 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:21.952041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:21.952043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:06:21.952122 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.952124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.953501 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.953502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.954710 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.954713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.955879 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.955880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.955966 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.955972 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:06:21.956391 (+ 419us) client_negotiation.cc:240] Negotiation successful
0619 12:06:21.956454 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5429,"thread_start_us":67,"threads_started":1}
I20260619 12:06:21.957413 12187 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed48520e57d41179b179373c91fa10a (DEFAULT_TABLE table=test-table [id=c8d54f41af6c4d6d9da26c65553eec0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:06:21.957779 12187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed48520e57d41179b179373c91fa10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:06:21.959314 12543 tablet_bootstrap.cc:492] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Bootstrap starting.
I20260619 12:06:21.960409 12543 tablet_bootstrap.cc:654] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Neither blocks nor log segments found. Creating new log.
I20260619 12:06:21.960721 12543 log.cc:824] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:06:21.961504 12543 tablet_bootstrap.cc:492] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: No bootstrap required, opened a new log
I20260619 12:06:21.961589 12543 ts_tablet_manager.cc:1397] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:06:21.963634 12543 raft_consensus.cc:348] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:21.963840 12543 raft_consensus.cc:374] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:06:21.963886 12543 raft_consensus.cc:729] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dee6007366a4d12afd84e3a1ee9da7f, State: Initialized, Role: FOLLOWER
I20260619 12:06:21.964003 12543 consensus_queue.cc:260] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:21.964295 12543 ts_tablet_manager.cc:1428] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:06:21.964489 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.945846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:46037 (local address 127.0.0.1:34238)
0619 12:06:21.947876 (+ 2030us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:21.947885 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:21.947921 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:21.957351 (+ 9430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:21.957352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:21.957355 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:06:21.957400 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.957401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.961673 (+ 4272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.961674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.962976 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:21.962980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.964193 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.964194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:21.964272 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.964280 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:06:21.964312 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 12:06:21.964371 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2010,"thread_start_us":67,"threads_started":1}
I20260619 12:06:21.964931 12525 heartbeater.cc:503] Master 127.0.203.190:36729 was elected leader, sending a full tablet report...
I20260619 12:06:21.965035 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:21.946031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.0.203.130:46037)
0619 12:06:21.957060 (+ 11029us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:21.957067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:21.957079 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:06:21.957127 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:21.957129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:21.957148 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:21.957243 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:06:21.957471 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.961555 (+ 4084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.963149 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:21.964085 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:21.964729 (+ 644us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:21.964847 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:21.964882 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:06:21.964949 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10968,"thread_start_us":90,"threads_started":1}
I20260619 12:06:21.965842 12324 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed48520e57d41179b179373c91fa10a (DEFAULT_TABLE table=test-table [id=c8d54f41af6c4d6d9da26c65553eec0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:06:21.966310 12324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed48520e57d41179b179373c91fa10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:06:21.966410 12545 tablet_bootstrap.cc:492] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e: Bootstrap starting.
I20260619 12:06:21.967842 12545 tablet_bootstrap.cc:654] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e: Neither blocks nor log segments found. Creating new log.
I20260619 12:06:21.968176 12545 log.cc:824] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:06:21.969112 12545 tablet_bootstrap.cc:492] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e: No bootstrap required, opened a new log
I20260619 12:06:21.969193 12545 ts_tablet_manager.cc:1397] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:06:21.973130 12545 raft_consensus.cc:348] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:21.973451 12545 raft_consensus.cc:374] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:06:21.973495 12545 raft_consensus.cc:729] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c03c1349da5476e95cd613a713cf81e, State: Initialized, Role: FOLLOWER
I20260619 12:06:21.974187 12545 consensus_queue.cc:260] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:21.974619 12545 ts_tablet_manager.cc:1428] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 12:06:21.974773 12253 heartbeater.cc:503] Master 127.0.203.190:36729 was elected leader, sending a full tablet report...
I20260619 12:06:21.974967 12548 tablet_bootstrap.cc:492] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Bootstrap starting.
I20260619 12:06:21.977324 12548 tablet_bootstrap.cc:654] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Neither blocks nor log segments found. Creating new log.
I20260619 12:06:21.977725 12548 log.cc:824] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Log is configured to *not* fsync() on all Append() calls
I20260619 12:06:21.978809 12548 tablet_bootstrap.cc:492] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: No bootstrap required, opened a new log
I20260619 12:06:21.978894 12548 ts_tablet_manager.cc:1397] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:06:21.981169 12548 raft_consensus.cc:348] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:21.981416 12548 raft_consensus.cc:374] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:06:21.981458 12548 raft_consensus.cc:729] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332e2ecd0b394ababdb83b423dd0a667, State: Initialized, Role: FOLLOWER
I20260619 12:06:21.982142 12548 consensus_queue.cc:260] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:21.982561 12548 ts_tablet_manager.cc:1428] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 12:06:21.982700 12390 heartbeater.cc:503] Master 127.0.203.190:36729 was elected leader, sending a full tablet report...
I20260619 12:06:22.072436 12551 raft_consensus.cc:482] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:06:22.072582 12551 raft_consensus.cc:504] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:22.073004 12551 leader_election.cc:288] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dee6007366a4d12afd84e3a1ee9da7f (127.0.203.131:44509), 3c03c1349da5476e95cd613a713cf81e (127.0.203.129:35215)
I20260619 12:06:22.078018 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:22.073286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44509 (local address 127.0.203.130:43309)
0619 12:06:22.073586 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:22.073598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:22.073656 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:22.073869 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:22.073870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:22.073872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:06:22.073948 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:22.073951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.074994 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.074995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:22.076360 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:22.076364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.077700 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.077701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:22.077800 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:22.077805 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:06:22.077874 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 12:06:22.077926 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
I20260619 12:06:22.078485 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:22.073383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:43309 (local address 127.0.203.131:44509)
0619 12:06:22.073505 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:22.073508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:22.073721 (+ 213us) server_negotiation.cc:407] Connection header received
0619 12:06:22.073757 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:22.073759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:22.073770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:22.073814 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:06:22.074017 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.074850 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.076900 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.077554 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.077626 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:22.077637 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:22.078365 (+ 728us) server_negotiation.cc:299] Negotiation successful
0619 12:06:22.078416 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 12:06:22.078939 12479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed48520e57d41179b179373c91fa10a" candidate_uuid: "332e2ecd0b394ababdb83b423dd0a667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" is_pre_election: true
I20260619 12:06:22.079187 12479 raft_consensus.cc:2454] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 332e2ecd0b394ababdb83b423dd0a667 in term 0.
I20260619 12:06:22.079667 12278 leader_election.cc:302] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332e2ecd0b394ababdb83b423dd0a667, 9dee6007366a4d12afd84e3a1ee9da7f; no voters:
I20260619 12:06:22.079891 12551 raft_consensus.cc:2781] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:06:22.079941 12551 raft_consensus.cc:482] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:06:22.079962 12551 raft_consensus.cc:3037] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:06:22.081094 12551 raft_consensus.cc:504] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:22.081260 12551 leader_election.cc:288] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [CANDIDATE]: Term 1 election: Requested vote from peers 9dee6007366a4d12afd84e3a1ee9da7f (127.0.203.131:44509), 3c03c1349da5476e95cd613a713cf81e (127.0.203.129:35215)
I20260619 12:06:22.081434 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:22.073411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35215 (local address 127.0.203.130:60401)
0619 12:06:22.078127 (+ 4716us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:22.078139 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:22.078197 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:22.078463 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:06:22.078465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:06:22.078468 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:06:22.078536 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:22.078538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.079432 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.079433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:22.080411 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:06:22.080414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.081208 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.081208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:06:22.081264 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:22.081268 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:06:22.081314 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 12:06:22.081364 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4701,"thread_start_us":124,"threads_started":1}
I20260619 12:06:22.081717 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:22.073597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:60401 (local address 127.0.203.129:35215)
0619 12:06:22.073647 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:22.073649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:22.078205 (+ 4556us) server_negotiation.cc:407] Connection header received
0619 12:06:22.078300 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:22.078302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:22.078318 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:22.078386 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:06:22.078616 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.079301 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.080515 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.081091 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.081163 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:22.081177 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:22.081592 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 12:06:22.081648 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:06:22.081931 12479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed48520e57d41179b179373c91fa10a" candidate_uuid: "332e2ecd0b394ababdb83b423dd0a667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dee6007366a4d12afd84e3a1ee9da7f"
I20260619 12:06:22.082019 12479 raft_consensus.cc:3037] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:06:22.082046 12204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed48520e57d41179b179373c91fa10a" candidate_uuid: "332e2ecd0b394ababdb83b423dd0a667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c03c1349da5476e95cd613a713cf81e" is_pre_election: true
I20260619 12:06:22.082262 12204 raft_consensus.cc:2454] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 332e2ecd0b394ababdb83b423dd0a667 in term 0.
I20260619 12:06:22.082406 12203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed48520e57d41179b179373c91fa10a" candidate_uuid: "332e2ecd0b394ababdb83b423dd0a667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c03c1349da5476e95cd613a713cf81e"
I20260619 12:06:22.082463 12203 raft_consensus.cc:3037] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:06:22.082968 12479 raft_consensus.cc:2454] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332e2ecd0b394ababdb83b423dd0a667 in term 1.
I20260619 12:06:22.083178 12278 leader_election.cc:302] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332e2ecd0b394ababdb83b423dd0a667, 9dee6007366a4d12afd84e3a1ee9da7f; no voters:
I20260619 12:06:22.083341 12551 raft_consensus.cc:2781] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:06:22.083537 12551 raft_consensus.cc:686] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 1 LEADER]: Becoming Leader. State: Replica: 332e2ecd0b394ababdb83b423dd0a667, State: Running, Role: LEADER
I20260619 12:06:22.083640 12551 consensus_queue.cc:237] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } }
I20260619 12:06:22.083738 12203 raft_consensus.cc:2454] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332e2ecd0b394ababdb83b423dd0a667 in term 1.
I20260619 12:06:22.084771 12041 catalog_manager.cc:5495] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 reported cstate change: term changed from 0 to 1, leader changed from <none> to 332e2ecd0b394ababdb83b423dd0a667 (127.0.203.130). New cstate: current_term: 1 leader_uuid: "332e2ecd0b394ababdb83b423dd0a667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "332e2ecd0b394ababdb83b423dd0a667" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 } health_report { overall_health: UNKNOWN } } }
Jun 19 12:06:22 dist-test-slave-44vj krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
W20260619 12:06:22.103157 12254 tablet.cc:2369] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:06:22.123202 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:22.119648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.0.203.130:46037)
0619 12:06:22.119743 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 12:06:22.119746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:06:22.119767 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:06:22.119834 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:06:22.119836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:06:22.119875 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:06:22.120030 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:06:22.120264 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.120998 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.122012 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:06:22.122311 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:06:22.122374 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:06:22.122603 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:06:22.122751 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:06:22.122828 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:06:22.123068 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 12:06:22.123094 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 12:06:22.125666 12391 tablet.cc:2369] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:06:22.126917 12203 raft_consensus.cc:1261] T 7ed48520e57d41179b179373c91fa10a P 3c03c1349da5476e95cd613a713cf81e [term 1 FOLLOWER]: Refusing update from remote peer 332e2ecd0b394ababdb83b423dd0a667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:06:22.127790 12553 consensus_queue.cc:1035] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c03c1349da5476e95cd613a713cf81e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:06:22.128669 12479 raft_consensus.cc:1261] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 1 FOLLOWER]: Refusing update from remote peer 332e2ecd0b394ababdb83b423dd0a667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:06:22.129063 12553 consensus_queue.cc:1035] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dee6007366a4d12afd84e3a1ee9da7f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:06:22.141417 12557 mvcc.cc:205] Tried to move back new op lower bound from 7298542723588812800 to 7298542723415355392. Current Snapshot: MvccSnapshot[applied={T|T < 7298542723588812800}]
I20260619 12:06:22.141606 12041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:06:22.141755 12041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:06:22.141757 12556 mvcc.cc:205] Tried to move back new op lower bound from 7298542723588812800 to 7298542723415355392. Current Snapshot: MvccSnapshot[applied={T|T < 7298542723588812800 or (T in {7298542723588812800})}]
I20260619 12:06:22.142884 12041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf448d7dbb22445db8599ec9316c9c46: Sending DeleteTablet for 3 replicas of tablet 7ed48520e57d41179b179373c91fa10a
I20260619 12:06:22.143563 12459 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ed48520e57d41179b179373c91fa10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:06:22 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:34608
I20260619 12:06:22.143859 12324 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ed48520e57d41179b179373c91fa10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:06:22 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:34238
I20260619 12:06:22.144091 12187 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ed48520e57d41179b179373c91fa10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:06:22 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:47340
I20260619 12:06:22.148033 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 12122
W20260619 12:06:22.150074 12526 tablet.cc:2369] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:06:22.153280 12559 mvcc.cc:205] Tried to move back new op lower bound from 7298542723588812800 to 7298542723415355392. Current Snapshot: MvccSnapshot[applied={T|T < 7298542723588812800}]
I20260619 12:06:22.159521 12564 tablet_replica.cc:330] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: stopping tablet replica
I20260619 12:06:22.159804 12565 tablet_replica.cc:330] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: stopping tablet replica
I20260619 12:06:22.167529 12565 raft_consensus.cc:2229] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:06:22.167901 12564 raft_consensus.cc:2229] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:06:22.168871 12564 raft_consensus.cc:2258] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:06:22.169656 12564 ts_tablet_manager.cc:1905] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:06:22.175527 12565 raft_consensus.cc:2258] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:06:22.176337 12565 ts_tablet_manager.cc:1905] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:06:22.181358 12564 ts_tablet_manager.cc:1918] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:06:22.182031 12565 ts_tablet_manager.cc:1918] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 12:06:22.182339 12027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:35215 (error 108)
I20260619 12:06:22.183135 12564 log.cc:1196] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/7ed48520e57d41179b179373c91fa10a
I20260619 12:06:22.183328 12565 log.cc:1196] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/7ed48520e57d41179b179373c91fa10a
I20260619 12:06:22.184069 12564 ts_tablet_manager.cc:1939] T 7ed48520e57d41179b179373c91fa10a P 9dee6007366a4d12afd84e3a1ee9da7f: Deleting consensus metadata
I20260619 12:06:22.184396 12565 ts_tablet_manager.cc:1939] T 7ed48520e57d41179b179373c91fa10a P 332e2ecd0b394ababdb83b423dd0a667: Deleting consensus metadata
I20260619 12:06:22.184651 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 12259
I20260619 12:06:22.186084 12026 catalog_manager.cc:4841] TS 9dee6007366a4d12afd84e3a1ee9da7f (127.0.203.131:44509): tablet 7ed48520e57d41179b179373c91fa10a (table test-table [id=c8d54f41af6c4d6d9da26c65553eec0e]) successfully deleted
I20260619 12:06:22.192049 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:06:22.191773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35215 (local address 127.0.0.1:47354)
0619 12:06:22.191858 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:22.191965 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:06:22.192332 12027 catalog_manager.cc:4584] TS 3c03c1349da5476e95cd613a713cf81e (127.0.203.129:35215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ed48520e57d41179b179373c91fa10a: Network error: Client connection negotiation failed: client connection to 127.0.203.129:35215: connect: Connection refused (error 111)
W20260619 12:06:22.197752 12537 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:06:22.197311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:46037 (local address 127.0.0.1:34258)
0619 12:06:22.197373 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:06:22.197386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:06:22.197429 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:06:22.197678 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.130:46037: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:06:22.207569 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 12396
I20260619 12:06:22.215699 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 12009
I20260619 12:06:22.221692 814 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 12:06:22.567781 814 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 12:06:22.567939 814 mini_ranger.cc:66] Stopping Ranger...
I20260619 12:06:22.903553 814 mini_ranger.cc:68] Stopped Ranger
2026-06-19 12:06:22.903 UTC [11595] LOG: received smart shutdown request
2026-06-19 12:06:22.906 UTC [11595] LOG: background worker "logical replication launcher" (PID 11604) exited with exit code 1
2026-06-19 12:06:22.907 UTC [11599] LOG: shutting down
2026-06-19 12:06:23.065 UTC [11595] LOG: database system is shut down
2026-06-19T12:06:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51360 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:06:23 dist-test-slave-44vj krb5kdc[12577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:06:23 dist-test-slave-44vj krb5kdc[12577](info): set up 2 sockets
Jun 19 12:06:23 dist-test-slave-44vj krb5kdc[12577](info): commencing operation
krb5kdc: starting...
W20260619 12:06:24.135213 814 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:06:24 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:06:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/rangeradmin_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/rangeradmin_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/rangerlookup_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/rangerlookup_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
I20260619 12:06:24.314374 814 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskgwdYux/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/postgres -l logfile start
2026-06-19 12:06:26.825 UTC [12607] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 12:06:26.825 UTC [12607] LOG: listening on IPv4 address "127.0.203.148", port 42481
2026-06-19 12:06:26.829 UTC [12607] LOG: listening on Unix socket "/tmp/.s.PGSQL.42481"
2026-06-19 12:06:26.848 UTC [12610] LOG: database system was shut down at 2026-06-19 12:06:25 UTC
2026-06-19 12:06:26.852 UTC [12607] LOG: database system is ready to accept connections
I20260619 12:06:27.739511 814 mini_postgres.cc:96] Postgres bound to 42481
2026-06-19 12:06:27.746 UTC [12618] FATAL: database "slave" does not exist
127.0.203.148:42481 - accepting connections
I20260619 12:06:27.746951 814 mini_ranger.cc:161] Starting Ranger...
I20260619 12:06:27.763200 814 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 12:06:28.219355 814 mini_ranger.cc:87] Created ranger Postgres database
I20260619 12:06:28.219486 814 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-admin
2026-06-19 12:06:28,508 [I] DB FLAVOR :POSTGRES
2026-06-19 12:06:28,509 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:06:28,509 [I] Checking connection..
2026-06-19 12:06:28,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 12:06:28,790 [I] Checking connection passed.
2026-06-19 12:06:28,790 [I] --------- Verifying version history table ---------
2026-06-19 12:06:28,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:06:29,071 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:06:29,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:06:29,335 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:06:29,336 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 12:06:29,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 12:06:29.608 UTC [12682] WARNING: there is no transaction in progress
2026-06-19 12:06:29,615 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 12:06:29,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:06:29,885 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 12:06:29,885 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:06:29,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 12:06:30,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 12:06:30,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-44vj.c.gcp-upstream.internal','N') ;"
2026-06-19 12:06:30,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:06:30,953 [I] Table x_portal_user does not exist in database ranger
2026-06-19 12:06:30,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:06:31,221 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 12:06:31,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:06:31,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:06:31,772 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 12:06:31,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 12:06:32.899 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:32.919 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:32.938 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:32.958 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:32.977 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.000 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.076 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.083 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.098 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.108 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.119 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.134 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.145 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.151 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.158 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33.166 UTC [12790] WARNING: there is no transaction in progress
2026-06-19 12:06:33,952 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 12:06:33,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 12:06:34,234 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 12:06:34,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:06:34,507 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 12:06:34,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:06:34,775 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 12:06:34,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:06:35,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:06:35,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-44vj.c.gcp-upstream.internal';"
2026-06-19 12:06:35,559 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 12:06:35,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 12:06:35,844 [I] DB_PATCHES have already been applied
I20260619 12:06:35.850327 814 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-admin:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskgwdYux/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 12:06:35.850405 814 mini_ranger.cc:193] Using host: 127.0.203.148
I20260619 12:06:35.853922 814 mini_ranger.cc:227] Ranger admin URL: http://127.0.203.148:48775
Jun 19, 2026 12:06:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:06:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:06:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 12:06:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 12:06:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 12:06:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 12:06:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 12:06:37.937851 814 mini_ranger.cc:160] Time spent starting Ranger: real 10.191s user 0.002s sys 0.009s
Jun 19, 2026 12:07:00 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 12:07:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 12:07:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 12:07:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 12:07:01 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 12:07:02.212339 814 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.203.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.203.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/rangerkms_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.203.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/rangerkms_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.203.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.203.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.203.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/HTTP_127.0.203.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 12:07:02 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870822, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 12:07:02.307546 814 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 12:07:02.318456 814 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 12:07:03.307557 814 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 12:07:03.307657 814 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms
I20260619 12:07:03.307673 814 mini_ranger_kms.cc:227] Using postgres at 127.0.203.148:42481
2026-06-19 12:07:03,476 [I] DB FLAVOR :POSTGRES
2026-06-19 12:07:03,476 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:07:03,476 [I] Checking connection
2026-06-19 12:07:03,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 12:07:03,742 [I] connection success
2026-06-19 12:07:03,743 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 12:07:03,743 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 12:07:03,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 12:07:04,020 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 12:07:04,021 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:07:04,021 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 12:07:04,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.203.148:42481/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 12:07:04,316 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 12:07:04.888716 814 mini_ranger_kms.cc:310] Created kms service
I20260619 12:07:05.141196 814 mini_ranger_kms.cc:326] Created kudu user
I20260619 12:07:05.194629 814 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 12:07:05 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870822, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:05.662775 814 mini_ranger_kms.cc:384] Added ranger policy
I20260619 12:07:05.663117 814 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms:/tmp/dist-test-taskgwdYux/build/release/bin/postgresql.jar:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskgwdYux/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 12:07:05.663156 814 mini_ranger_kms.cc:240] Using host: 127.0.203.148
I20260619 12:07:05.666481 814 mini_ranger_kms.cc:276] Ranger KMS PID: 12972
I20260619 12:07:05.666561 814 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.203.148:44925
Jun 19, 2026 12:07:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:07:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:07:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 12:07:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 12:07:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 12:07:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 12:07:07.655050 814 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.348s user 0.008s sys 0.144s
I20260619 12:07:07.655170 814 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 12:07:07.655229 814 mini_ranger_kms.cc:401] 127.0.203.148:44925/kms/v1/keys
Jun 19 12:07:08 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781870828, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.203.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 12:07:08 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870828, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.203.148@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
Jun 19 12:07:09 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870828, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.203.148@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
Jun 19 12:07:11 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:07:11.651057 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43811
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:43811
--ranger_config_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:11.745251 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:11.745549 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:11.745574 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:11.747054 13022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:11.747115 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:11.747129 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:11.747141 13022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:11.747152 13022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:11.749130 13022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.203.148:44925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:43811
--ranger_config_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:11.749467 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:11.749737 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:07:11.755578 13022 server_base.cc:1034] running on GCE node
W20260619 12:07:11.755546 13030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:11.755749 13028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:11.756431 13027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:11.756871 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:11.757221 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:11.758401 13022 hybrid_clock.cc:648] HybridClock initialized: now 1781870831758371 us; error 55 us; skew 500 ppm
Jun 19 12:07:11 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:11.761813 13022 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:11.762394 13022 webserver.cc:460] Webserver started at http://127.0.203.190:46403/ using document root <none> and password file <none>
I20260619 12:07:11.762639 13022 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:11.762686 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:11.762836 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:07:11 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:11.821800 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "e33dab24c5f6445a9368a3a9f58c82b9"
format_stamp: "Formatted at 2026-06-19 12:07:11 on dist-test-slave-44vj"
server_key: "7e13bd397657c3bdd97d3e690a87679e"
server_key_iv: "dea0735d2bc83d0a912a075989d04972"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:11.822266 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "e33dab24c5f6445a9368a3a9f58c82b9"
format_stamp: "Formatted at 2026-06-19 12:07:11 on dist-test-slave-44vj"
server_key: "7e13bd397657c3bdd97d3e690a87679e"
server_key_iv: "dea0735d2bc83d0a912a075989d04972"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:11.824019 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 12:07:11.840718 13037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:11.841423 13022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:11.841595 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "e33dab24c5f6445a9368a3a9f58c82b9"
format_stamp: "Formatted at 2026-06-19 12:07:11 on dist-test-slave-44vj"
server_key: "7e13bd397657c3bdd97d3e690a87679e"
server_key_iv: "dea0735d2bc83d0a912a075989d04972"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:11.841725 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:11.861593 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:11.864136 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:11.864320 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:11.870353 13022 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:43811
I20260619 12:07:11.870484 13089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:43811 every 8 connection(s)
I20260619 12:07:11.870950 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:11.871848 13090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:11.874819 13090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Bootstrap starting.
I20260619 12:07:11.875597 13090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:11.875911 13090 log.cc:824] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:11.877002 13090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: No bootstrap required, opened a new log
I20260619 12:07:11.878859 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13022
I20260619 12:07:11.878983 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:07:11.879180 814 external_mini_cluster.cc:1428] Setting key 543997135c7de997f357144320ad4db4
I20260619 12:07:11.879725 13090 raft_consensus.cc:348] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:11.880064 13090 raft_consensus.cc:374] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:11.880128 13090 raft_consensus.cc:729] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33dab24c5f6445a9368a3a9f58c82b9, State: Initialized, Role: FOLLOWER
I20260619 12:07:11.880327 13090 consensus_queue.cc:260] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:11.880590 13090 raft_consensus.cc:388] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:07:11.880642 13090 raft_consensus.cc:482] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:07:11.880692 13090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:11.881683 13090 raft_consensus.cc:504] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:11.881814 13090 leader_election.cc:302] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e33dab24c5f6445a9368a3a9f58c82b9; no voters:
I20260619 12:07:11.882036 13090 leader_election.cc:288] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:07:11.882133 13095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:11.882450 13095 raft_consensus.cc:686] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 LEADER]: Becoming Leader. State: Replica: e33dab24c5f6445a9368a3a9f58c82b9, State: Running, Role: LEADER
I20260619 12:07:11.882495 13090 sys_catalog.cc:564] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:11.884176 13095 consensus_queue.cc:237] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:11.885011 13096 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } } }
I20260619 12:07:11.885109 13096 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:11.885123 13090 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:07:11.885488 13097 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e33dab24c5f6445a9368a3a9f58c82b9. Latest consensus state: current_term: 1 leader_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } } }
I20260619 12:07:11.885576 13109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:11.885591 13097 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:11.886245 13109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 12:07:11 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:11.892051 13109 catalog_manager.cc:1347] Generated new cluster ID: f522d7441c8c40e0a1b530b4c3114e21
I20260619 12:07:11.892201 13109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:07:11.904331 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:11.880236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45294 (local address 127.0.203.190:43811)
0619 12:07:11.880496 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:11.880510 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:11.880564 (+ 54us) server_negotiation.cc:407] Connection header received
0619 12:07:11.880733 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:11.880741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:11.880810 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:11.880986 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:11.881864 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:11.883227 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:11.884303 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:11.885715 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:11.887842 (+ 2127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:11.890279 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:11.890345 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:11.890346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:11.890475 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:11.901471 (+ 10996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:11.902072 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:11.902076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:11.902076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:11.902212 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:11.902438 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:11.902438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:11.902439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:11.902833 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:11.903157 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:11.903876 (+ 719us) server_negotiation.cc:299] Negotiation successful
0619 12:07:11.904055 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":131,"threads_started":1}
I20260619 12:07:11.912390 13109 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:07:11.913339 13109 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:07:11.925863 13109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Generated new TSK 0
I20260619 12:07:11.926270 13109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:07:12.754788 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43811
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:12.854121 13135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:12.854346 13135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:12.854382 13135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:12.855721 13135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:12.855768 13135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:12.855806 13135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:07:12.857421 13135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.203.148:44925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:12.857689 13135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:12.857928 13135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:12.863233 13143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:12.863227 13141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:12.863227 13140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:12.864434 13135 server_base.cc:1034] running on GCE node
I20260619 12:07:12.864694 13135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:12.864945 13135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:12.866118 13135 hybrid_clock.cc:648] HybridClock initialized: now 1781870832866103 us; error 46 us; skew 500 ppm
Jun 19 12:07:12 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870832, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:12.869195 13135 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:07:12.869673 13135 webserver.cc:460] Webserver started at http://127.0.203.129:44815/ using document root <none> and password file <none>
I20260619 12:07:12.869884 13135 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:12.869931 13135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:12.870059 13135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:07:12 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870832, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:12.891615 13135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5"
format_stamp: "Formatted at 2026-06-19 12:07:12 on dist-test-slave-44vj"
server_key: "b649df28ac09ccc43ec2b41932147190"
server_key_iv: "f1cf0d4939d5bba0cf7e21a73de71258"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:12.892099 13135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5"
format_stamp: "Formatted at 2026-06-19 12:07:12 on dist-test-slave-44vj"
server_key: "b649df28ac09ccc43ec2b41932147190"
server_key_iv: "f1cf0d4939d5bba0cf7e21a73de71258"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:12.895136 13135 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:12.911520 13150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:12.911748 13135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:07:12.911861 13135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5"
format_stamp: "Formatted at 2026-06-19 12:07:12 on dist-test-slave-44vj"
server_key: "b649df28ac09ccc43ec2b41932147190"
server_key_iv: "f1cf0d4939d5bba0cf7e21a73de71258"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:12.911996 13135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:12.935711 13135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:12.938426 13135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:12.938688 13135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:12.938999 13135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:12.939524 13135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:12.939625 13135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:12.939739 13135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:12.939827 13135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:12.948628 13135 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:43929
I20260619 12:07:12.949215 13135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:07:12.954049 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13135
I20260619 12:07:12.954151 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:07:12.954360 814 external_mini_cluster.cc:1428] Setting key 9c63f5028623e6ee14e89e33183e5bba
I20260619 12:07:12.960340 13265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:43929 every 8 connection(s)
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Jun 19 12:07:12 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870832, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:12.979907 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:12.960667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43811 (local address 127.0.203.129:48673)
0619 12:07:12.962492 (+ 1825us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:12.962551 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:12.963247 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:12.966045 (+ 2798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:12.966052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:12.966354 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:12.966666 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:12.966674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:12.967722 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:12.967723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:12.969042 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:12.969046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:12.969641 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:12.969642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:12.969802 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:12.970563 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:12.970575 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:12.974853 (+ 4278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:12.977916 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:12.977927 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:12.977930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:12.978309 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:12.978582 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:12.978583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:12.978583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:12.978748 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:12.979100 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:12.979107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:12.979294 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 12:07:12.979450 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 12:07:12.979642 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1763,"thread_start_us":157,"threads_started":1}
I20260619 12:07:12.980610 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:12.961119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:48673 (local address 127.0.203.190:43811)
0619 12:07:12.965718 (+ 4599us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:12.965721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:12.965735 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:07:12.965766 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:12.965768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:12.965782 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:12.965868 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:12.966790 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:12.967612 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:12.969152 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:12.969481 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:12.969550 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:12.974992 (+ 5442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:12.974995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:12.974995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:12.975004 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:12.977789 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:12.978416 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:12.978417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:12.978418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:12.978497 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:12.978824 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:12.978825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:12.978825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:12.978985 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:12.979059 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:12.980395 (+ 1336us) server_negotiation.cc:299] Negotiation successful
0619 12:07:12.980502 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260619 12:07:12.981524 13266 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43811
I20260619 12:07:12.981635 13266 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:12.981851 13266 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:12.982532 13054 ts_manager.cc:194] Registered new tserver with Master: 09a64a1f7b9c47c082b8a0f79fd4f5e5 (127.0.203.129:43929)
I20260619 12:07:12.983503 13054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:48673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:07:12.994680 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43811
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:13.119454 13272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:13.119711 13272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:13.119748 13272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:13.121276 13272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:13.121335 13272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:13.121374 13272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:07:13.123219 13272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.203.148:44925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:13.123539 13272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:13.123828 13272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:13.129513 13282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:13.129599 13279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:13.129843 13272 server_base.cc:1034] running on GCE node
W20260619 12:07:13.129513 13280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:13.130227 13272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:13.130523 13272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:13.131692 13272 hybrid_clock.cc:648] HybridClock initialized: now 1781870833131665 us; error 35 us; skew 500 ppm
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:13.135512 13272 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:07:13.136246 13272 webserver.cc:460] Webserver started at http://127.0.203.130:45603/ using document root <none> and password file <none>
I20260619 12:07:13.136541 13272 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:13.136617 13272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:13.136785 13272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:13.174633 13272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "2082229f0f5f7ac12bffaf7257b28802"
server_key_iv: "6a69887f35f1366f72406b9c45e4192c"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.175174 13272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "2082229f0f5f7ac12bffaf7257b28802"
server_key_iv: "6a69887f35f1366f72406b9c45e4192c"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.179862 13272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260619 12:07:13.196014 13289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.196379 13272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:13.196497 13272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "2082229f0f5f7ac12bffaf7257b28802"
server_key_iv: "6a69887f35f1366f72406b9c45e4192c"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.196640 13272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:13.211757 13272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:13.214550 13272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:13.214792 13272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:13.215139 13272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:13.215667 13272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:13.215763 13272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.215847 13272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:13.215909 13272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.223737 13272 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:45241
I20260619 12:07:13.224360 13272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:07:13.229570 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13272
I20260619 12:07:13.229676 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:07:13.229893 814 external_mini_cluster.cc:1428] Setting key 0aa808b5257550eb01d585587d98a228
I20260619 12:07:13.240348 13403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:45241 every 8 connection(s)
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:13.256894 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:13.241696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:34799 (local address 127.0.203.190:43811)
0619 12:07:13.241943 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:13.241946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:13.242017 (+ 71us) server_negotiation.cc:407] Connection header received
0619 12:07:13.242651 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:13.242652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:13.242672 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:13.242726 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:13.243638 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.244425 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.246150 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.246473 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.246523 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:13.252996 (+ 6473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:13.252999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:13.252999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:13.253010 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:13.254833 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:13.255479 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:13.255481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:13.255481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:13.255551 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:13.255876 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:13.255878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:13.255878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:13.256043 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:13.256101 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:13.256611 (+ 510us) server_negotiation.cc:299] Negotiation successful
0619 12:07:13.256758 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260619 12:07:13.256964 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:13.241332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43811 (local address 127.0.203.130:34799)
0619 12:07:13.241872 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:13.241911 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:13.242546 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:13.242876 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:13.242881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:13.243142 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:13.243522 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:13.243529 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.244577 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.244578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:13.246053 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:13.246057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.246644 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.246645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:13.246790 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:13.250778 (+ 3988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:13.250794 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:13.252816 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:13.254970 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:13.254983 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:13.254987 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:13.255352 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:13.255642 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:13.255642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:13.255643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:13.255794 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:13.256142 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:13.256150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:13.256350 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 12:07:13.256475 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 12:07:13.256682 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":116,"threads_started":1}
I20260619 12:07:13.257545 13404 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43811
I20260619 12:07:13.257690 13404 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:13.257930 13404 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:13.258539 13054 ts_manager.cc:194] Registered new tserver with Master: 329ad4f20d6f4a478bfa4b22ab5d4dea (127.0.203.130:45241)
I20260619 12:07:13.259213 13054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:34799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:07:13.266747 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43811
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:13.420145 13409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:13.420553 13409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:13.420661 13409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:13.422609 13409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:13.422757 13409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:13.422845 13409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:07:13.425351 13409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.203.148:44925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:13.425908 13409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:13.426257 13409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:13.433379 13414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:13.433768 13415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:13.436146 13417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:13.441051 13409 server_base.cc:1034] running on GCE node
I20260619 12:07:13.441466 13409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:13.441897 13409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:13.443081 13409 hybrid_clock.cc:648] HybridClock initialized: now 1781870833443055 us; error 52 us; skew 500 ppm
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:13.447419 13409 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:07:13.448194 13409 webserver.cc:460] Webserver started at http://127.0.203.131:45837/ using document root <none> and password file <none>
I20260619 12:07:13.448607 13409 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:13.448771 13409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:13.449000 13409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:13.468343 13409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "d2761af6db2e461dab6bc4d191991e8d"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "4e1dd802dd63ddcf7d4f916ed4efc6ab"
server_key_iv: "8416a73152ef3a6b2992d0adc4c7ffbf"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.468812 13409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "d2761af6db2e461dab6bc4d191991e8d"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "4e1dd802dd63ddcf7d4f916ed4efc6ab"
server_key_iv: "8416a73152ef3a6b2992d0adc4c7ffbf"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.470352 13409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:07:13.489312 13424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.489652 13409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:07:13.489758 13409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "d2761af6db2e461dab6bc4d191991e8d"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "4e1dd802dd63ddcf7d4f916ed4efc6ab"
server_key_iv: "8416a73152ef3a6b2992d0adc4c7ffbf"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.489890 13409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:13.514376 13409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:13.517514 13409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:13.517748 13409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:13.518132 13409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:13.518616 13409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:13.518708 13409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.518803 13409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:13.518862 13409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.526419 13409 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:45279
I20260619 12:07:13.527007 13409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:07:13.529541 13537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:45279 every 8 connection(s)
I20260619 12:07:13.534011 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13409
I20260619 12:07:13.534114 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:07:13.534324 814 external_mini_cluster.cc:1428] Setting key 6437f228f749f7e55765bb44fec5ec81
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:13.544827 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:13.530338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:54457 (local address 127.0.203.190:43811)
0619 12:07:13.530397 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:13.530399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:13.530952 (+ 553us) server_negotiation.cc:407] Connection header received
0619 12:07:13.531804 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:13.531806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:13.531822 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:13.531929 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:13.532977 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.533793 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.537075 (+ 3282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.537405 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.537518 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:13.540614 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:13.540617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:13.540617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:13.540630 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:13.542787 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:13.543577 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:13.543578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:13.543579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:13.543648 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:13.543984 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:13.543985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:13.543985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:13.544161 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:13.544220 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:13.544627 (+ 407us) server_negotiation.cc:299] Negotiation successful
0619 12:07:13.544721 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:07:13.545504 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:13.530307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43811 (local address 127.0.203.131:54457)
0619 12:07:13.530770 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:13.530836 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:13.531676 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:13.532214 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:13.532220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:13.532510 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:13.532867 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:13.532875 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.535547 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.535548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:13.536965 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:13.536970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:13.537555 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:13.537556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:13.537726 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:13.538488 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:13.538501 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:13.540481 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:13.542949 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:13.542967 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:13.542970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:13.543437 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:13.543730 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:13.543730 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:13.543731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:13.543894 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:13.544259 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:13.544266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:13.544456 (+ 190us) client_negotiation.cc:769] Sending connection context
0619 12:07:13.545028 (+ 572us) client_negotiation.cc:240] Negotiation successful
0619 12:07:13.545235 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":117,"threads_started":1}
I20260619 12:07:13.546082 13538 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43811
I20260619 12:07:13.546200 13538 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:13.546427 13538 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:13.547519 13054 ts_manager.cc:194] Registered new tserver with Master: d2761af6db2e461dab6bc4d191991e8d (127.0.203.131:45279)
I20260619 12:07:13.548038 13054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:54457
I20260619 12:07:13.558163 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:07:13.559028 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13135
I20260619 12:07:13.566910 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13272
I20260619 12:07:13.573830 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13409
I20260619 12:07:13.582479 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13022
I20260619 12:07:13.589937 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43811
--webserver_interface=127.0.203.190
--webserver_port=46403
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:43811
--ranger_config_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:13.679005 13541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:13.679260 13541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:13.679298 13541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:13.680614 13541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:13.680667 13541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:13.680687 13541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:13.680702 13541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:13.680717 13541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:13.682425 13541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:43811
--ranger_config_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=46403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:13.682693 13541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:13.682919 13541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:13.688119 13546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:13.688100 13547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:13.688436 13549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:13.689218 13541 server_base.cc:1034] running on GCE node
I20260619 12:07:13.689520 13541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:13.689884 13541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:13.691071 13541 hybrid_clock.cc:648] HybridClock initialized: now 1781870833691041 us; error 57 us; skew 500 ppm
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:13.694519 13541 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:13.695060 13541 webserver.cc:460] Webserver started at http://127.0.203.190:46403/ using document root <none> and password file <none>
I20260619 12:07:13.695318 13541 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:13.695415 13541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:13.778584 13541 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:13.779840 13557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.780145 13541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:13.780315 13541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "e33dab24c5f6445a9368a3a9f58c82b9"
format_stamp: "Formatted at 2026-06-19 12:07:11 on dist-test-slave-44vj"
server_key: "7e13bd397657c3bdd97d3e690a87679e"
server_key_iv: "dea0735d2bc83d0a912a075989d04972"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.780916 13541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:13.795655 13541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:13.797586 13541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:13.797719 13541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:13.802664 13541 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:43811
I20260619 12:07:13.802722 13609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:43811 every 8 connection(s)
I20260619 12:07:13.803151 13541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:13.804800 13610 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:07:13.805560 13610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Bootstrap starting.
I20260619 12:07:13.807982 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13541
I20260619 12:07:13.808275 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:43929
--local_ip_for_outbound_sockets=127.0.203.129
--tserver_master_addrs=127.0.203.190:43811
--webserver_port=44815
--webserver_interface=127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:13.815781 13610 log.cc:824] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:13.817847 13610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:07:13.818127 13610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Bootstrap complete.
I20260619 12:07:13.820641 13610 raft_consensus.cc:348] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:13.820919 13610 raft_consensus.cc:729] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33dab24c5f6445a9368a3a9f58c82b9, State: Initialized, Role: FOLLOWER
I20260619 12:07:13.821096 13610 consensus_queue.cc:260] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:13.821172 13610 raft_consensus.cc:388] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:07:13.821223 13610 raft_consensus.cc:482] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:07:13.821271 13610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:07:13.822479 13610 raft_consensus.cc:504] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:13.822594 13610 leader_election.cc:302] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e33dab24c5f6445a9368a3a9f58c82b9; no voters:
I20260619 12:07:13.822798 13610 leader_election.cc:288] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:07:13.822853 13614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:07:13.822979 13614 raft_consensus.cc:686] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [term 2 LEADER]: Becoming Leader. State: Replica: e33dab24c5f6445a9368a3a9f58c82b9, State: Running, Role: LEADER
I20260619 12:07:13.823153 13610 sys_catalog.cc:564] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:13.823189 13614 consensus_queue.cc:237] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } }
I20260619 12:07:13.823664 13615 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } } }
I20260619 12:07:13.823747 13615 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:13.824040 13616 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e33dab24c5f6445a9368a3a9f58c82b9. Latest consensus state: current_term: 2 leader_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33dab24c5f6445a9368a3a9f58c82b9" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43811 } } }
I20260619 12:07:13.824154 13616 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:13.824868 13610 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:07:13.827543 13620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:13.828804 13620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:07:13.829089 13620 catalog_manager.cc:1259] Loaded cluster ID: f522d7441c8c40e0a1b530b4c3114e21
I20260619 12:07:13.829275 13620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:07:13.830792 13620 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:07:13.831101 13620 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Loaded TSK: 0
I20260619 12:07:13.831444 13620 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:07:13.924737 13612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:13.925004 13612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:13.925146 13612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:13.927003 13612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:13.927085 13612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:13.927129 13612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:07:13.929132 13612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:43929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=44815
--tserver_master_addrs=127.0.203.190:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:13.929443 13612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:13.929728 13612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:13.934795 13650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:13.934967 13648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:13.935213 13612 server_base.cc:1034] running on GCE node
W20260619 12:07:13.935252 13647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:13.935636 13612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:13.935926 13612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:13.941025 13612 hybrid_clock.cc:648] HybridClock initialized: now 1781870833940980 us; error 53 us; skew 500 ppm
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:13.944169 13612 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:07:13.944641 13612 webserver.cc:460] Webserver started at http://127.0.203.129:44815/ using document root <none> and password file <none>
I20260619 12:07:13.944859 13612 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:13.944926 13612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:07:13 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:13.984624 13612 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:13.985613 13658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:13.985764 13612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:07:13.985845 13612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5"
format_stamp: "Formatted at 2026-06-19 12:07:12 on dist-test-slave-44vj"
server_key: "b649df28ac09ccc43ec2b41932147190"
server_key_iv: "f1cf0d4939d5bba0cf7e21a73de71258"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:13.986413 13612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:14.010618 13612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:14.013417 13612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:14.013597 13612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:14.013868 13612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:14.014243 13612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:14.014279 13612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.014322 13612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:14.014349 13612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.022154 13612 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:43929
I20260619 12:07:14.022301 13771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:43929 every 8 connection(s)
I20260619 12:07:14.022670 13612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:07:14.025476 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13612
I20260619 12:07:14.025779 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:45241
--local_ip_for_outbound_sockets=127.0.203.130
--tserver_master_addrs=127.0.203.190:43811
--webserver_port=45603
--webserver_interface=127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:07:14 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:14.045017 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:14.023725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43811 (local address 127.0.203.129:34769)
0619 12:07:14.023971 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:14.024040 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:14.024781 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:14.032256 (+ 7475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:14.032264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:14.032631 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:14.033013 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:14.033021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.034982 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.034984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:14.036529 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:14.036535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.037583 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.037584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:14.037728 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:14.038401 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:14.038422 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:14.040355 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:14.042852 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:14.042860 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:14.042863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:14.043187 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:14.043546 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:14.043547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:14.043547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:14.043697 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:14.044233 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:14.044240 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:14.044404 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 12:07:14.044518 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:07:14.044713 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":120,"threads_started":1}
I20260619 12:07:14.045812 13775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:14.023896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:34769 (local address 127.0.203.190:43811)
0619 12:07:14.031521 (+ 7625us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:14.031537 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:14.031602 (+ 65us) server_negotiation.cc:407] Connection header received
0619 12:07:14.031801 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:14.031810 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:14.031880 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:14.033184 (+ 1304us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:14.033291 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.034802 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.036706 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.037426 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.037590 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:14.040530 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:14.040537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:14.040537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:14.040555 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:14.042739 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.043295 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.043298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.043299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.043465 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.043789 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.043789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.043790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.044149 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:14.045375 (+ 1226us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:14.045463 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 12:07:14.045602 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6278,"thread_start_us":95,"threads_started":1}
I20260619 12:07:14.046530 13772 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43811
I20260619 12:07:14.046643 13772 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.046867 13772 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
W20260619 12:07:14.047818 13772 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:43811 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:07:14.047901 13772 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.047976 13772 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:14.048280 13772 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.048358 13772 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
W20260619 12:07:14.048868 13772 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:07:14.144347 13776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:14.144686 13776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:14.144728 13776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:14.146946 13776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:14.147058 13776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:14.147105 13776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:07:14.149132 13776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:45241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=45603
--tserver_master_addrs=127.0.203.190:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:14.149411 13776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:14.149683 13776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:14.154984 13784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:14.154996 13782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:14.155563 13781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:14.157032 13776 server_base.cc:1034] running on GCE node
I20260619 12:07:14.157300 13776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:14.157572 13776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:14.158769 13776 hybrid_clock.cc:648] HybridClock initialized: now 1781870834158752 us; error 25 us; skew 500 ppm
Jun 19 12:07:14 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:14.165859 13776 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:07:14.166584 13776 webserver.cc:460] Webserver started at http://127.0.203.130:45603/ using document root <none> and password file <none>
I20260619 12:07:14.166908 13776 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:14.167001 13776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:07:14 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:14.197733 13776 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:07:14.198792 13792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.199156 13776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:14.199326 13776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "2082229f0f5f7ac12bffaf7257b28802"
server_key_iv: "6a69887f35f1366f72406b9c45e4192c"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:14.199827 13776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:14.233244 13776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:14.235652 13776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:14.235842 13776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:14.236146 13776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:14.236536 13776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:14.236577 13776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.236603 13776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:14.236639 13776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.244642 13776 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:45241
I20260619 12:07:14.245078 13776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:07:14.245720 13905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:45241 every 8 connection(s)
I20260619 12:07:14.250980 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13776
I20260619 12:07:14.251320 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.203.148:44925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:45279
--local_ip_for_outbound_sockets=127.0.203.131
--tserver_master_addrs=127.0.203.190:43811
--webserver_port=45837
--webserver_interface=127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:07:14 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:14.257400 13775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:14.246253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:37443 (local address 127.0.203.190:43811)
0619 12:07:14.246309 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:14.246311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:14.246323 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:07:14.246785 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:14.246786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:14.246805 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:14.246865 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:14.247499 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.248090 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.249120 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.249341 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.249408 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:14.253623 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:14.253625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:14.253626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:14.253636 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:14.255174 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.256158 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.256159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.256160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.256241 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.256609 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.256610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.256611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.256746 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:14.256813 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:14.257155 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 12:07:14.257288 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:07:14.257827 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:14.245889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43811 (local address 127.0.203.130:37443)
0619 12:07:14.246060 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:14.246102 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:14.246654 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:14.246936 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:14.246940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:14.247148 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:14.247347 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:14.247352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.248181 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.248183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:14.249015 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:14.249019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.249443 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.249445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:14.249591 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:14.250178 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:14.250186 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:14.253473 (+ 3287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:14.255336 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:14.255354 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:14.255393 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:14.255997 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:14.256350 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:14.256352 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:14.256352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:14.256527 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:14.256810 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:14.256819 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:14.256988 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 12:07:14.257107 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 12:07:14.257310 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260619 12:07:14.258347 13906 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43811
I20260619 12:07:14.258448 13906 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.258649 13906 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
W20260619 12:07:14.259325 13906 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:43811 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:07:14.259435 13906 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.259518 13906 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:14.259850 13906 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.259933 13906 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
W20260619 12:07:14.260237 13906 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:07:14.374179 13909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:14.374454 13909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:14.374495 13909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:14.376119 13909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:14.376183 13909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:14.376230 13909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:07:14.378228 13909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.203.148:44925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:45279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=45837
--tserver_master_addrs=127.0.203.190:43811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:14.378551 13909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:14.378830 13909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:14.385054 13915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:14.385425 13918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:14.385529 13916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:14.386970 13909 server_base.cc:1034] running on GCE node
I20260619 12:07:14.387310 13909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:14.387861 13909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:14.389097 13909 hybrid_clock.cc:648] HybridClock initialized: now 1781870834389067 us; error 53 us; skew 500 ppm
Jun 19 12:07:14 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:14.393149 13909 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:07:14.393949 13909 webserver.cc:460] Webserver started at http://127.0.203.131:45837/ using document root <none> and password file <none>
I20260619 12:07:14.394322 13909 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:14.394431 13909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:07:14 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:14.426810 13909 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:07:14.428671 13926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.429040 13909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:14.429204 13909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "d2761af6db2e461dab6bc4d191991e8d"
format_stamp: "Formatted at 2026-06-19 12:07:13 on dist-test-slave-44vj"
server_key: "4e1dd802dd63ddcf7d4f916ed4efc6ab"
server_key_iv: "8416a73152ef3a6b2992d0adc4c7ffbf"
server_key_version: "kuduclusterkey@0"
I20260619 12:07:14.429857 13909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:14.447198 13909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:14.449944 13909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:14.450085 13909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:14.450395 13909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:14.450845 13909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:14.450894 13909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.450920 13909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:14.450932 13909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:14.461170 13909 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:45279
I20260619 12:07:14.461942 13909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:07:14.462462 14039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:45279 every 8 connection(s)
I20260619 12:07:14.467641 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 13909
Jun 19 12:07:14 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:14.476460 13775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:14.463826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:33647 (local address 127.0.203.190:43811)
0619 12:07:14.463888 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:14.463890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:14.464212 (+ 322us) server_negotiation.cc:407] Connection header received
0619 12:07:14.465016 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:14.465017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:14.465034 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:14.465091 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:14.465956 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.467020 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.469146 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.469462 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.469527 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:14.472374 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:14.472376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:14.472377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:14.472387 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:14.474311 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.474861 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.474863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.474863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.474936 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.475327 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.475330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.475331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.475635 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:14.475743 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:14.476220 (+ 477us) server_negotiation.cc:299] Negotiation successful
0619 12:07:14.476356 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:07:14.476719 14042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:14.463474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43811 (local address 127.0.203.131:33647)
0619 12:07:14.463998 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:14.464053 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:14.464875 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:14.465186 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:14.465190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:14.465455 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:14.465822 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:14.465831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.467166 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.467167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:14.468995 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:14.469001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.469646 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.469648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:14.469792 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:14.470508 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:14.470521 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:14.472268 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:14.474420 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:14.474432 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:14.474436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:14.474770 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:14.475010 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:14.475011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:14.475012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:14.475169 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:14.475740 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:14.475749 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:14.475979 (+ 230us) client_negotiation.cc:769] Sending connection context
0619 12:07:14.476126 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 12:07:14.476403 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":95,"threads_started":1}
I20260619 12:07:14.477474 14040 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43811
I20260619 12:07:14.477609 14040 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.477880 14040 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
W20260619 12:07:14.478930 14040 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:43811 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:07:14.479020 14040 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.479105 14040 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:14.479614 14040 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:14.479773 14040 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
W20260619 12:07:14.480228 14040 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:07:14.480333 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:14.468155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.0.203.190:43811)
0619 12:07:14.470309 (+ 2154us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:14.470311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:14.470318 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:07:14.470341 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:14.470342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:14.470356 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:14.470423 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:14.472648 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.473396 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.474858 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:14.475165 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:14.475242 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:14.477170 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:14.477172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:14.477173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:14.477181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:14.479026 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.479468 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.479470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.479470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.479543 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:14.479727 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:14.479728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:14.479728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:14.479873 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:14.480080 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:14.480128 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 12:07:14.480222 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2131,"thread_start_us":129,"threads_started":1}
I20260619 12:07:15.049152 13772 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:15.049618 13772 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:15.050355 13574 ts_manager.cc:194] Registered new tserver with Master: 09a64a1f7b9c47c082b8a0f79fd4f5e5 (127.0.203.129:43929)
I20260619 12:07:15.051220 13574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:34769
Jun 19 12:07:15 dist-test-slave-44vj krb5kdc[12577](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781870835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:15.260411 13906 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:15.260598 13906 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:15.261125 13574 ts_manager.cc:194] Registered new tserver with Master: 329ad4f20d6f4a478bfa4b22ab5d4dea (127.0.203.130:45241)
I20260619 12:07:15.261734 13574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:37443
I20260619 12:07:15.480451 14040 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:15.480657 14040 heartbeater.cc:511] Master 127.0.203.190:43811 requested a full tablet report, sending...
I20260619 12:07:15.481194 13574 ts_manager.cc:194] Registered new tserver with Master: d2761af6db2e461dab6bc4d191991e8d (127.0.203.131:45279)
I20260619 12:07:15.481813 13574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:33647
I20260619 12:07:15.482064 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:07:15.496567 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.485776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.0.203.190:43811)
0619 12:07:15.485943 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:15.485945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:15.486105 (+ 160us) server_negotiation.cc:407] Connection header received
0619 12:07:15.486209 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:15.486210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:15.486228 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:15.486328 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:15.487178 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.488045 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.489214 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.489550 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.489626 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.491578 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:15.491580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:15.491580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:15.491603 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:15.494755 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:15.495254 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:15.495255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:15.495256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:15.495321 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:15.495552 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:15.495552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:15.495553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:15.495720 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:15.495818 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:15.496341 (+ 523us) server_negotiation.cc:299] Negotiation successful
0619 12:07:15.496448 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260619 12:07:15.499764 13573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:07:15.502628 13573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:07:15.516959 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.509571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:43929 (local address 127.0.0.1:37686)
0619 12:07:15.510100 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:15.510155 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:15.510242 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:15.511834 (+ 1592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:15.511835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:15.511837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:15.511917 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.511919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.513337 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.513339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.514851 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.514856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.516507 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.516509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.516656 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.516680 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:07:15.516755 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:07:15.516834 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":89,"thread_start_us":140,"threads_started":1}
I20260619 12:07:15.517836 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.509667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.0.203.129:43929)
0619 12:07:15.511493 (+ 1826us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:15.511499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:15.511509 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:15.511549 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:15.511551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:15.511568 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:15.511981 (+ 413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:15.512067 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.513217 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.514987 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.516391 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.516604 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.517648 (+ 1044us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:15.517682 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 12:07:15.517745 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1781,"thread_start_us":64,"threads_started":1}
I20260619 12:07:15.518177 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.509902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:45241 (local address 127.0.0.1:43824)
0619 12:07:15.511706 (+ 1804us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:15.511717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:15.511761 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:15.513831 (+ 2070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:15.513832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:15.513835 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:15.513890 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.513892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.515234 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.515236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.516645 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.516650 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.517873 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.517875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.517954 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.517962 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:07:15.517997 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 12:07:15.518057 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1783,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260619 12:07:15.518602 14072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.510444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.0.203.130:45241)
0619 12:07:15.513659 (+ 3215us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:15.513665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:15.513675 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:15.513712 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:15.513714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:15.513732 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:15.513953 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:15.514022 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.515117 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.516795 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.517739 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.518346 (+ 607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.518430 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:15.518464 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 12:07:15.518511 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3170,"thread_start_us":82,"threads_started":1}
I20260619 12:07:15.518935 13706 tablet_service.cc:1467] Processing CreateTablet for tablet a7d07af623b343819072dcc24d68ef05 (DEFAULT_TABLE table=test-table [id=2b228d2fb81144d49196dbda64b082e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:15.519287 13840 tablet_service.cc:1467] Processing CreateTablet for tablet a7d07af623b343819072dcc24d68ef05 (DEFAULT_TABLE table=test-table [id=2b228d2fb81144d49196dbda64b082e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:15.519446 13706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d07af623b343819072dcc24d68ef05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:15.519680 13840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d07af623b343819072dcc24d68ef05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:15.521579 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.510151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.0.203.131:45279)
0619 12:07:15.510853 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:15.510860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:15.511042 (+ 182us) server_negotiation.cc:407] Connection header received
0619 12:07:15.511104 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:15.511108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:15.511128 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:15.511227 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:15.511554 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.512730 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.520138 (+ 7408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.520994 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.521057 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.521147 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:15.521424 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 12:07:15.521484 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":65,"threads_started":1}
I20260619 12:07:15.522226 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.509576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:45279 (local address 127.0.0.1:51668)
0619 12:07:15.510925 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:15.510936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:15.511016 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:15.511263 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:15.511267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:15.511270 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:15.511443 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.511446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.513229 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.513230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.519998 (+ 6768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.520003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.521225 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.521226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.521317 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.521323 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:07:15.522074 (+ 751us) client_negotiation.cc:240] Negotiation successful
0619 12:07:15.522141 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1326,"mutex_wait_us":58,"thread_start_us":156,"threads_started":1}
I20260619 12:07:15.523130 13974 tablet_service.cc:1467] Processing CreateTablet for tablet a7d07af623b343819072dcc24d68ef05 (DEFAULT_TABLE table=test-table [id=2b228d2fb81144d49196dbda64b082e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:15.523634 13974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d07af623b343819072dcc24d68ef05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:15.524274 14073 tablet_bootstrap.cc:492] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Bootstrap starting.
I20260619 12:07:15.525488 14073 tablet_bootstrap.cc:654] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:15.525949 14073 log.cc:824] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:15.527720 14074 tablet_bootstrap.cc:492] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5: Bootstrap starting.
I20260619 12:07:15.528650 14074 tablet_bootstrap.cc:654] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:15.529068 14074 log.cc:824] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:15.530123 14073 tablet_bootstrap.cc:492] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: No bootstrap required, opened a new log
I20260619 12:07:15.530207 14073 ts_tablet_manager.cc:1397] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 12:07:15.530210 14074 tablet_bootstrap.cc:492] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5: No bootstrap required, opened a new log
I20260619 12:07:15.530330 14074 ts_tablet_manager.cc:1397] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:15.532869 14073 raft_consensus.cc:348] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.533087 14073 raft_consensus.cc:374] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:15.533120 14073 raft_consensus.cc:729] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329ad4f20d6f4a478bfa4b22ab5d4dea, State: Initialized, Role: FOLLOWER
I20260619 12:07:15.533459 14073 consensus_queue.cc:260] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.533852 14073 ts_tablet_manager.cc:1428] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 12:07:15.534015 13906 heartbeater.cc:503] Master 127.0.203.190:43811 was elected leader, sending a full tablet report...
I20260619 12:07:15.533897 14074 raft_consensus.cc:348] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.534104 14074 raft_consensus.cc:374] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:15.534135 14074 raft_consensus.cc:729] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a64a1f7b9c47c082b8a0f79fd4f5e5, State: Initialized, Role: FOLLOWER
I20260619 12:07:15.534401 14074 consensus_queue.cc:260] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.534731 14074 ts_tablet_manager.cc:1428] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 12:07:15.534858 13772 heartbeater.cc:503] Master 127.0.203.190:43811 was elected leader, sending a full tablet report...
I20260619 12:07:15.535413 14076 tablet_bootstrap.cc:492] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Bootstrap starting.
I20260619 12:07:15.536222 14076 tablet_bootstrap.cc:654] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:15.536625 14076 log.cc:824] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:15.537865 14076 tablet_bootstrap.cc:492] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: No bootstrap required, opened a new log
I20260619 12:07:15.537998 14076 ts_tablet_manager.cc:1397] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:15.540953 14076 raft_consensus.cc:348] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.541246 14076 raft_consensus.cc:374] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:15.541321 14076 raft_consensus.cc:729] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2761af6db2e461dab6bc4d191991e8d, State: Initialized, Role: FOLLOWER
I20260619 12:07:15.541522 14076 consensus_queue.cc:260] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.541908 14076 ts_tablet_manager.cc:1428] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260619 12:07:15.542083 14040 heartbeater.cc:503] Master 127.0.203.190:43811 was elected leader, sending a full tablet report...
I20260619 12:07:15.560266 14081 raft_consensus.cc:482] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:15.562127 14081 raft_consensus.cc:504] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.562744 14081 leader_election.cc:288] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09a64a1f7b9c47c082b8a0f79fd4f5e5 (127.0.203.129:43929), 329ad4f20d6f4a478bfa4b22ab5d4dea (127.0.203.130:45241)
I20260619 12:07:15.569047 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.563010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:47545 (local address 127.0.203.129:43929)
0619 12:07:15.563081 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:15.563083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:15.563244 (+ 161us) server_negotiation.cc:407] Connection header received
0619 12:07:15.563409 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:15.563411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:15.563423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:15.563459 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:15.563766 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.565117 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.567154 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.568129 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.568184 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.568216 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:15.568888 (+ 672us) server_negotiation.cc:299] Negotiation successful
0619 12:07:15.568955 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:07:15.570266 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.562927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:43929 (local address 127.0.203.131:47545)
0619 12:07:15.563175 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:15.563187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:15.563276 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:15.563547 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:15.563548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:15.563550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:15.563669 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.563672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.565248 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.565249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.566992 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.566997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.568693 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.568695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.568802 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.568807 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:07:15.569664 (+ 857us) client_negotiation.cc:240] Negotiation successful
0619 12:07:15.570136 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":125,"threads_started":1}
I20260619 12:07:15.570837 13726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d07af623b343819072dcc24d68ef05" candidate_uuid: "d2761af6db2e461dab6bc4d191991e8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" is_pre_election: true
I20260619 12:07:15.571028 13726 raft_consensus.cc:2454] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2761af6db2e461dab6bc4d191991e8d in term 0.
I20260619 12:07:15.571424 13929 leader_election.cc:302] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a64a1f7b9c47c082b8a0f79fd4f5e5, d2761af6db2e461dab6bc4d191991e8d; no voters:
I20260619 12:07:15.571570 14081 raft_consensus.cc:2781] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:07:15.571605 14081 raft_consensus.cc:482] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:07:15.571624 14081 raft_consensus.cc:3037] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:15.572620 14081 raft_consensus.cc:504] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.572762 14081 leader_election.cc:288] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [CANDIDATE]: Term 1 election: Requested vote from peers 09a64a1f7b9c47c082b8a0f79fd4f5e5 (127.0.203.129:43929), 329ad4f20d6f4a478bfa4b22ab5d4dea (127.0.203.130:45241)
I20260619 12:07:15.573167 13726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d07af623b343819072dcc24d68ef05" candidate_uuid: "d2761af6db2e461dab6bc4d191991e8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5"
I20260619 12:07:15.573268 13726 raft_consensus.cc:3037] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:15.574222 13726 raft_consensus.cc:2454] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2761af6db2e461dab6bc4d191991e8d in term 1.
I20260619 12:07:15.574493 13929 leader_election.cc:302] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a64a1f7b9c47c082b8a0f79fd4f5e5, d2761af6db2e461dab6bc4d191991e8d; no voters:
I20260619 12:07:15.574635 14081 raft_consensus.cc:2781] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:15.574807 14072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.563496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:56349 (local address 127.0.203.130:45241)
0619 12:07:15.563553 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:15.563555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:15.570612 (+ 7057us) server_negotiation.cc:407] Connection header received
0619 12:07:15.570734 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:15.570735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:15.570746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:15.570772 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:15.570943 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.571641 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.573550 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.574173 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.574229 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.574242 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:15.574673 (+ 431us) server_negotiation.cc:299] Negotiation successful
0619 12:07:15.574741 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:07:15.574810 14081 raft_consensus.cc:686] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 1 LEADER]: Becoming Leader. State: Replica: d2761af6db2e461dab6bc4d191991e8d, State: Running, Role: LEADER
I20260619 12:07:15.574915 14081 consensus_queue.cc:237] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } }
I20260619 12:07:15.574980 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.563252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:45241 (local address 127.0.203.131:56349)
0619 12:07:15.570536 (+ 7284us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:15.570548 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:15.570648 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:15.570837 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:15.570837 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:15.570839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:15.570878 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.570879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.571751 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.571751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.573441 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:15.573444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.574525 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.574526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:15.574590 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.574594 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:07:15.574876 (+ 282us) client_negotiation.cc:240] Negotiation successful
0619 12:07:15.574923 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7268,"thread_start_us":90,"threads_started":1}
I20260619 12:07:15.575827 13573 catalog_manager.cc:5495] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d reported cstate change: term changed from 0 to 1, leader changed from <none> to d2761af6db2e461dab6bc4d191991e8d (127.0.203.131). New cstate: current_term: 1 leader_uuid: "d2761af6db2e461dab6bc4d191991e8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2761af6db2e461dab6bc4d191991e8d" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260619 12:07:15.577531 13860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d07af623b343819072dcc24d68ef05" candidate_uuid: "d2761af6db2e461dab6bc4d191991e8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" is_pre_election: true
I20260619 12:07:15.577692 13860 raft_consensus.cc:2454] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2761af6db2e461dab6bc4d191991e8d in term 0.
I20260619 12:07:15.577796 13859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d07af623b343819072dcc24d68ef05" candidate_uuid: "d2761af6db2e461dab6bc4d191991e8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea"
I20260619 12:07:15.577839 13859 raft_consensus.cc:3037] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:15.578711 13859 raft_consensus.cc:2454] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2761af6db2e461dab6bc4d191991e8d in term 1.
Jun 19 12:07:15 dist-test-slave-44vj krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for HTTP/127.0.203.148@KRBTEST.COM
I20260619 12:07:15.591233 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.587606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51672 (local address 127.0.203.131:45279)
0619 12:07:15.587662 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:15.587664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:15.587678 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:07:15.587741 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:15.587742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:15.587773 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:15.587909 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:07:15.588149 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.588928 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.590074 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:15.590446 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:15.590576 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:15.590772 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:07:15.590935 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:07:15.591027 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:15.591079 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 12:07:15.591110 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:07:15.594007 13859 raft_consensus.cc:1261] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 1 FOLLOWER]: Refusing update from remote peer d2761af6db2e461dab6bc4d191991e8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:15.594151 13726 raft_consensus.cc:1261] T a7d07af623b343819072dcc24d68ef05 P 09a64a1f7b9c47c082b8a0f79fd4f5e5 [term 1 FOLLOWER]: Refusing update from remote peer d2761af6db2e461dab6bc4d191991e8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:15.594864 14081 consensus_queue.cc:1035] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09a64a1f7b9c47c082b8a0f79fd4f5e5" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:15.595074 14081 consensus_queue.cc:1035] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "329ad4f20d6f4a478bfa4b22ab5d4dea" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:15.601660 14087 mvcc.cc:205] Tried to move back new op lower bound from 7298542942589263872 to 7298542942515384320. Current Snapshot: MvccSnapshot[applied={T|T < 7298542942589263872}]
I20260619 12:07:15.603080 14090 mvcc.cc:205] Tried to move back new op lower bound from 7298542942589263872 to 7298542942515384320. Current Snapshot: MvccSnapshot[applied={T|T < 7298542942589263872}]
I20260619 12:07:15.603816 14088 mvcc.cc:205] Tried to move back new op lower bound from 7298542942589263872 to 7298542942515384320. Current Snapshot: MvccSnapshot[applied={T|T < 7298542942589263872}]
I20260619 12:07:15.607568 13573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:15.607731 13573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:15.608593 13573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e33dab24c5f6445a9368a3a9f58c82b9: Sending DeleteTablet for 3 replicas of tablet a7d07af623b343819072dcc24d68ef05
I20260619 12:07:15.609117 13840 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d07af623b343819072dcc24d68ef05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:15 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:43824
I20260619 12:07:15.609159 13706 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d07af623b343819072dcc24d68ef05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:15 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:37686
I20260619 12:07:15.609730 13974 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d07af623b343819072dcc24d68ef05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:15 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:51668
I20260619 12:07:15.610025 14099 tablet_replica.cc:330] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: stopping tablet replica
I20260619 12:07:15.610110 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13612
I20260619 12:07:15.610167 14099 raft_consensus.cc:2229] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:07:15.613765 14099 raft_consensus.cc:2258] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:15.614529 14099 ts_tablet_manager.cc:1905] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:07:15.616655 14099 ts_tablet_manager.cc:1918] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:15.616823 14099 log.cc:1196] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/a7d07af623b343819072dcc24d68ef05
I20260619 12:07:15.617187 14099 ts_tablet_manager.cc:1939] T a7d07af623b343819072dcc24d68ef05 P d2761af6db2e461dab6bc4d191991e8d: Deleting consensus metadata
I20260619 12:07:15.617872 13558 catalog_manager.cc:4841] TS d2761af6db2e461dab6bc4d191991e8d (127.0.203.131:45279): tablet a7d07af623b343819072dcc24d68ef05 (table test-table [id=2b228d2fb81144d49196dbda64b082e3]) successfully deleted
I20260619 12:07:15.623575 14097 tablet_replica.cc:330] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: stopping tablet replica
I20260619 12:07:15.623821 14097 raft_consensus.cc:2229] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:07:15.624019 14097 raft_consensus.cc:2258] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:07:15.624354 13560 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:43929 (error 108)
I20260619 12:07:15.625102 14097 ts_tablet_manager.cc:1905] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:07:15.627240 14097 ts_tablet_manager.cc:1918] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:15.627334 14097 log.cc:1196] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/a7d07af623b343819072dcc24d68ef05
I20260619 12:07:15.627663 14097 ts_tablet_manager.cc:1939] T a7d07af623b343819072dcc24d68ef05 P 329ad4f20d6f4a478bfa4b22ab5d4dea: Deleting consensus metadata
I20260619 12:07:15.628237 13560 catalog_manager.cc:4841] TS 329ad4f20d6f4a478bfa4b22ab5d4dea (127.0.203.130:45241): tablet a7d07af623b343819072dcc24d68ef05 (table test-table [id=2b228d2fb81144d49196dbda64b082e3]) successfully deleted
I20260619 12:07:15.628513 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13776
I20260619 12:07:15.633203 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:15.632983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:43929 (local address 127.0.0.1:37702)
0619 12:07:15.633069 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:15.633137 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 12:07:15.633420 13560 catalog_manager.cc:4584] TS 09a64a1f7b9c47c082b8a0f79fd4f5e5 (127.0.203.129:43929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7d07af623b343819072dcc24d68ef05: Network error: Client connection negotiation failed: client connection to 127.0.203.129:43929: connect: Connection refused (error 111)
I20260619 12:07:15.655586 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13909
I20260619 12:07:15.662195 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 13541
I20260619 12:07:15.670336 814 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 12:07:16.015635 814 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 12:07:16.015761 814 mini_ranger.cc:66] Stopping Ranger...
I20260619 12:07:16.352166 814 mini_ranger.cc:68] Stopped Ranger
2026-06-19 12:07:16.352 UTC [12607] LOG: received smart shutdown request
2026-06-19 12:07:16.356 UTC [12607] LOG: background worker "logical replication launcher" (PID 12616) exited with exit code 1
2026-06-19 12:07:16.357 UTC [12611] LOG: shutting down
2026-06-19 12:07:16.526 UTC [12607] LOG: database system is shut down
2026-06-19T12:07:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53466 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:07:16 dist-test-slave-44vj krb5kdc[14109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:07:16 dist-test-slave-44vj krb5kdc[14109](info): set up 2 sockets
Jun 19 12:07:16 dist-test-slave-44vj krb5kdc[14109](info): commencing operation
krb5kdc: starting...
W20260619 12:07:17.599460 814 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:07:17 dist-test-slave-44vj krb5kdc[14109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:07:17.736074 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33075
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33075,127.0.203.189:33133,127.0.203.188:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:17.824918 14122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:17.825163 14122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:17.825188 14122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:17.826464 14122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:17.826529 14122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:17.826542 14122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:17.826555 14122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:17.826566 14122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:17.828516 14122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33075,127.0.203.189:33133,127.0.203.188:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:17.828861 14122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:17.829150 14122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:17.836282 14130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:17.836199 14127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:17.836220 14128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:17.836452 14122 server_base.cc:1034] running on GCE node
I20260619 12:07:17.837029 14122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:17.837505 14122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:17.838680 14122 hybrid_clock.cc:648] HybridClock initialized: now 1781870837838651 us; error 59 us; skew 500 ppm
Jun 19 12:07:17 dist-test-slave-44vj krb5kdc[14109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:17.841753 14122 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:17.842355 14122 webserver.cc:460] Webserver started at http://127.0.203.190:33935/ using document root <none> and password file <none>
I20260619 12:07:17.842607 14122 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:17.842658 14122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:17.842808 14122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:17.844115 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "27cde3ba9db64f899efeae199e6aa393"
format_stamp: "Formatted at 2026-06-19 12:07:17 on dist-test-slave-44vj"
server_key: "d9df9a72d28df5517974d2cb69f55853"
server_key_iv: "fd46a797c16970ec1ce4e86b58ca91f0"
server_key_version: "encryptionkey@0"
I20260619 12:07:17.844511 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "27cde3ba9db64f899efeae199e6aa393"
format_stamp: "Formatted at 2026-06-19 12:07:17 on dist-test-slave-44vj"
server_key: "d9df9a72d28df5517974d2cb69f55853"
server_key_iv: "fd46a797c16970ec1ce4e86b58ca91f0"
server_key_version: "encryptionkey@0"
I20260619 12:07:17.846144 14122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:17.847136 14137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:17.847340 14122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:17.847488 14122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "27cde3ba9db64f899efeae199e6aa393"
format_stamp: "Formatted at 2026-06-19 12:07:17 on dist-test-slave-44vj"
server_key: "d9df9a72d28df5517974d2cb69f55853"
server_key_iv: "fd46a797c16970ec1ce4e86b58ca91f0"
server_key_version: "encryptionkey@0"
I20260619 12:07:17.847565 14122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:17.860318 14122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:17.862744 14122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:17.862881 14122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:17.867772 14122 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:33075
I20260619 12:07:17.867920 14189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:33075 every 8 connection(s)
I20260619 12:07:17.868244 14122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:17.869326 14190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:17.871076 14190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:07:17.872051 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14122
I20260619 12:07:17.872149 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:07:17.872336 814 external_mini_cluster.cc:1428] Setting key f3f5b058f8a7df7b535ef8e143df7279
Jun 19 12:07:17 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.189@KRBTEST.COM" created.
I20260619 12:07:17.883183 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:17.871936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33075 (local address 127.0.0.1:32812)
0619 12:07:17.872184 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:17.872240 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:17.872928 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:17.873289 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:17.873293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:17.873663 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:17.873921 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:17.873927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:17.875432 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:17.875433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:17.876601 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:17.876606 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:17.877263 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:17.877264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:17.877378 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:17.878019 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:17.878031 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:17.879856 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:17.881656 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:17.881662 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:17.881665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:17.881868 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:17.882092 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:17.882093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:17.882093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:17.882180 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:17.882543 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:17.882547 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:17.882616 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 12:07:17.882705 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 12:07:17.882956 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":81,"thread_start_us":109,"threads_started":1}
I20260619 12:07:17.883180 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:17.871965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.0.203.190:33075)
0619 12:07:17.872194 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:17.872206 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:17.872371 (+ 165us) server_negotiation.cc:407] Connection header received
0619 12:07:17.873119 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:17.873122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:17.873145 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:17.873248 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:17.874055 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:17.875269 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:17.876727 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:17.877145 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:17.877251 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:17.879946 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:17.879955 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:17.879955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:17.879963 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:17.881570 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:17.881941 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:17.881945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:17.881945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:17.882027 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:17.882233 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:17.882234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:17.882234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:17.882483 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:17.882559 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:17.882732 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 12:07:17.882895 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":186,"thread_start_us":103,"threads_started":1}
I20260619 12:07:17.884320 14190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:07:17.884786 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:17.884635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56796)
0619 12:07:17.884696 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:17.884758 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:07:17.884938 14141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
I20260619 12:07:17.885361 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:17.885279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56806)
0619 12:07:17.885317 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:17.885342 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 12:07:17.885452 14190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:33133: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab.
Entry for principal kudu/127.0.203.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab.
I20260619 12:07:17.899139 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:33133
--webserver_interface=127.0.203.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab
--principal=kudu/127.0.203.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33075,127.0.203.189:33133,127.0.203.188:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:17.934746 14190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } attempt: 1
I20260619 12:07:17.935282 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:17.935158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56808)
0619 12:07:17.935220 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:17.935253 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 12:07:17.935827 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:17.935699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56810)
0619 12:07:17.935774 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:17.935802 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 12:07:17.935931 14190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:33133: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
W20260619 12:07:17.990415 14197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:17.990759 14197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:17.990799 14197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:17.992378 14197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:17.992453 14197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:17.992475 14197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:17.992492 14197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:17.992507 14197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:17.994415 14197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33075,127.0.203.189:33133,127.0.203.188:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab
--principal=kudu/127.0.203.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:33133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.203.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:17.994728 14197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:17.994992 14197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:18.000639 14203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:18.000772 14197 server_base.cc:1034] running on GCE node
W20260619 12:07:18.000669 14205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:18.000643 14202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:18.001211 14197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:18.001569 14197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:18.002738 14197 hybrid_clock.cc:648] HybridClock initialized: now 1781870838002702 us; error 59 us; skew 500 ppm
I20260619 12:07:18.004087 14190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } attempt: 2
I20260619 12:07:18.004554 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.004421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56812)
0619 12:07:18.004475 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.004526 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:07:18.005021 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.004904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56820)
0619 12:07:18.004969 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.004999 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 12:07:18.005198 14190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:33133: Network error: Client connection negotiation failed: client connection to 127.0.203.189:33133: connect: Connection refused (error 111)
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:18.005918 14197 init.cc:377] Logged in from keytab as kudu/127.0.203.189@KRBTEST.COM (short username kudu)
I20260619 12:07:18.006439 14197 webserver.cc:460] Webserver started at http://127.0.203.189:41277/ using document root <none> and password file <none>
I20260619 12:07:18.006646 14197 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:18.006691 14197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:18.006791 14197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:18.008025 14197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data/instance:
uuid: "2b9e12f923084c0f8f63c1821c19795a"
format_stamp: "Formatted at 2026-06-19 12:07:18 on dist-test-slave-44vj"
server_key: "14615533c1d2dfa7c62c39ac798d8a11"
server_key_iv: "6a128309d82493dc806bd08f4fa985c4"
server_key_version: "encryptionkey@0"
I20260619 12:07:18.008368 14197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/wal/instance:
uuid: "2b9e12f923084c0f8f63c1821c19795a"
format_stamp: "Formatted at 2026-06-19 12:07:18 on dist-test-slave-44vj"
server_key: "14615533c1d2dfa7c62c39ac798d8a11"
server_key_iv: "6a128309d82493dc806bd08f4fa985c4"
server_key_version: "encryptionkey@0"
I20260619 12:07:18.010005 14197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:18.010905 14212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:18.011281 14197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:18.011423 14197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/wal
uuid: "2b9e12f923084c0f8f63c1821c19795a"
format_stamp: "Formatted at 2026-06-19 12:07:18 on dist-test-slave-44vj"
server_key: "14615533c1d2dfa7c62c39ac798d8a11"
server_key_iv: "6a128309d82493dc806bd08f4fa985c4"
server_key_version: "encryptionkey@0"
I20260619 12:07:18.011529 14197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:18.026075 14197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:18.028589 14197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:18.028723 14197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:18.033080 14197 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.189:33133
I20260619 12:07:18.033156 14264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.189:33133 every 8 connection(s)
I20260619 12:07:18.033524 14197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
I20260619 12:07:18.034684 14265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:18.035205 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14197
I20260619 12:07:18.035329 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-1/wal/instance
I20260619 12:07:18.035537 814 external_mini_cluster.cc:1428] Setting key 3e4b7f19ebf8f58dec06138653a7a03b
I20260619 12:07:18.037225 14265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.203.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.188@KRBTEST.COM" created.
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:18.051622 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.038274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.0.203.190:33075)
0619 12:07:18.038334 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.038385 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.038641 (+ 256us) server_negotiation.cc:407] Connection header received
0619 12:07:18.039193 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.039194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.039207 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.039269 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.040094 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.040649 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.041737 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.041964 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.042025 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.047289 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.047291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.047291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.047303 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.049595 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.050126 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.050127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.050128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.050213 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.050657 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.050658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.050659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.050840 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.050928 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.051341 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.051505 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:07:18.051676 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.038267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33075 (local address 127.0.0.1:32820)
0619 12:07:18.038490 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.038534 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.039085 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.039384 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.039389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.039698 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.039994 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.040000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.040736 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.040737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.041660 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.041663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.042037 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.042039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.042175 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.042763 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.042773 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.047115 (+ 4342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.049722 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.049731 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.049734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.050036 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.050373 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.050375 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.050375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.050547 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.050924 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.050931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.051095 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.051201 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.051410 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":124,"threads_started":1}
I20260619 12:07:18.052438 14265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:07:18.061311 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.052732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56822)
0619 12:07:18.052775 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.052786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.052824 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.053238 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.053240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.053449 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.053556 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.053561 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.054611 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.054612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.055455 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.055459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.055974 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.055976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.056064 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.056508 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.056514 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.058076 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.059977 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.059979 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.059980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.060251 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.060500 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.060501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.060501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.060570 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.060867 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.060868 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.060962 (+ 94us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.061040 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.061164 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 12:07:18.061335 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.052850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.0.203.189:33133)
0619 12:07:18.053045 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.053050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.053070 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:07:18.053089 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.053091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.053103 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.053173 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.053705 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.054481 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.055550 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.055894 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.055954 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.058198 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.058207 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.058208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.058216 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.059861 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.060354 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.060357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.060357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.060427 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.060630 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.060631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.060631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.060788 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.060853 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.061070 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.061217 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":166,"thread_start_us":123,"threads_started":1}
I20260619 12:07:18.062268 14265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:07:18.062801 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.062551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32816)
0619 12:07:18.062665 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.062767 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260619 12:07:18.062995 14215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
I20260619 12:07:18.063665 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.063501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32830)
0619 12:07:18.063597 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.063640 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260619 12:07:18.063941 14265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:43609: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab.
Entry for principal kudu/127.0.203.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab.
I20260619 12:07:18.064868 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:43609
--webserver_interface=127.0.203.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab
--principal=kudu/127.0.203.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33075,127.0.203.189:33133,127.0.203.188:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:18.096400 14190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } attempt: 3
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:07:18.104127 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.097006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.0.203.189:33133)
0619 12:07:18.097101 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.097104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.097114 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:18.097140 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.097141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.097153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.097199 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.097550 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.098074 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.098744 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.098969 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.099026 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.101151 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.101153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.101154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.101162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.102654 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.103161 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.103163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.103163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.103228 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.103489 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.103489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.103490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.103611 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.103671 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.103936 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.104035 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:07:18.104182 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.096910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56824)
0619 12:07:18.096972 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.096992 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.097042 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.097210 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.097211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.097401 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.097474 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.097476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.098151 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.098152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.098670 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.098671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.099032 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.099033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.099104 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.099593 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.099597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.100980 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.102783 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.102785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.102786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.103052 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.103307 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.103308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.103308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.103414 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.103684 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.103685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.103768 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.103876 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.104047 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 12:07:18.104789 14190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:07:18.105254 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.105120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32836)
0619 12:07:18.105183 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.105228 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:07:18.105624 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.105533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32850)
0619 12:07:18.105581 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.105607 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 12:07:18.105831 14190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:43609: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
I20260619 12:07:18.113253 14265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } attempt: 1
I20260619 12:07:18.113924 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.113729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32862)
0619 12:07:18.113838 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.113893 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 12:07:18.114385 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.114280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32870)
0619 12:07:18.114342 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.114366 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:07:18.114481 14265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:43609: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
I20260619 12:07:18.137010 14190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } attempt: 1
I20260619 12:07:18.137737 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.137525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32884)
0619 12:07:18.137613 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.137693 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:07:18.138355 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.138159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32894)
0619 12:07:18.138253 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.138310 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260619 12:07:18.138505 14190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:43609: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
W20260619 12:07:18.157711 14272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:18.157965 14272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:18.157989 14272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:18.159184 14272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:18.159225 14272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:18.159238 14272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:18.159250 14272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:18.159260 14272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:18.160964 14272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33075,127.0.203.189:33133,127.0.203.188:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab
--principal=kudu/127.0.203.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:43609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.203.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:18.161203 14272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:18.161420 14272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:18.166510 14278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:18.166625 14277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:18.166664 14280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:18.167023 14272 server_base.cc:1034] running on GCE node
I20260619 12:07:18.167339 14272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:18.167726 14272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:18.168922 14272 hybrid_clock.cc:648] HybridClock initialized: now 1781870838168894 us; error 41 us; skew 500 ppm
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:18.172086 14272 init.cc:377] Logged in from keytab as kudu/127.0.203.188@KRBTEST.COM (short username kudu)
I20260619 12:07:18.172681 14272 webserver.cc:460] Webserver started at http://127.0.203.188:44243/ using document root <none> and password file <none>
I20260619 12:07:18.172916 14272 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:18.172969 14272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:18.173122 14272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:18.174295 14272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data/instance:
uuid: "7a71fc0f012540ceb9c38b8eeb7801f3"
format_stamp: "Formatted at 2026-06-19 12:07:18 on dist-test-slave-44vj"
server_key: "f343d0e981ec41c6114f0dcbf15c71e4"
server_key_iv: "479e71e955326ed641c19dfe73098bb2"
server_key_version: "encryptionkey@0"
I20260619 12:07:18.174672 14272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/wal/instance:
uuid: "7a71fc0f012540ceb9c38b8eeb7801f3"
format_stamp: "Formatted at 2026-06-19 12:07:18 on dist-test-slave-44vj"
server_key: "f343d0e981ec41c6114f0dcbf15c71e4"
server_key_iv: "479e71e955326ed641c19dfe73098bb2"
server_key_version: "encryptionkey@0"
I20260619 12:07:18.176204 14272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:07:18.177081 14287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:18.177286 14272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:18.177397 14272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/wal
uuid: "7a71fc0f012540ceb9c38b8eeb7801f3"
format_stamp: "Formatted at 2026-06-19 12:07:18 on dist-test-slave-44vj"
server_key: "f343d0e981ec41c6114f0dcbf15c71e4"
server_key_iv: "479e71e955326ed641c19dfe73098bb2"
server_key_version: "encryptionkey@0"
I20260619 12:07:18.177465 14272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:18.182642 14265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } attempt: 2
I20260619 12:07:18.183233 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.183028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32902)
0619 12:07:18.183108 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.183184 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 12:07:18.183674 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.183582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32914)
0619 12:07:18.183628 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.183658 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 12:07:18.183777 14265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:43609: Network error: Client connection negotiation failed: client connection to 127.0.203.188:43609: connect: Connection refused (error 111)
I20260619 12:07:18.194696 14272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:18.196846 14272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:18.196967 14272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:18.201200 14272 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.188:43609
I20260619 12:07:18.201264 14339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.188:43609 every 8 connection(s)
I20260619 12:07:18.201614 14272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
I20260619 12:07:18.202669 14340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:18.204578 14340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:18.210965 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14272
I20260619 12:07:18.211063 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870673623844-814-0/minicluster-data/master-2/wal/instance
I20260619 12:07:18.211233 814 external_mini_cluster.cc:1428] Setting key d969fac3abc66bec3b6527e1db765bce
I20260619 12:07:18.213675 14190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } attempt: 2
I20260619 12:07:18.215569 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.205460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.0.203.190:33075)
0619 12:07:18.205567 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.205570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.205809 (+ 239us) server_negotiation.cc:407] Connection header received
0619 12:07:18.206283 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.206284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.206298 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.206368 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.207093 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.207601 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.208673 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.208885 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.208954 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.211708 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.211709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.211710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.211718 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.213894 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.214471 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.214472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.214473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.214533 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.214855 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.214855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.214856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.214994 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.215049 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.215350 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.215472 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 12:07:18.215785 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.205400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33075 (local address 127.0.0.1:32834)
0619 12:07:18.205607 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.205650 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.206191 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.206443 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.206447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.206730 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.206985 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.206989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.207681 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.207682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.208566 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.208569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.208990 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.208992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.209119 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.209684 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.209693 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.211571 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.214004 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.214017 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.214020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.214393 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.214602 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.214603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.214603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.214779 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.215053 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.215060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.215221 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.215318 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.215540 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260619 12:07:18.216518 14340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:07:18.222235 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.211694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.0.203.188:43609)
0619 12:07:18.211863 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.211868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.211913 (+ 45us) server_negotiation.cc:407] Connection header received
0619 12:07:18.212010 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.212013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.212030 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.212119 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.212912 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.213839 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.214901 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.215437 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.215506 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.218436 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.218445 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.218445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.218452 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.220576 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.220949 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.220954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.220954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.221045 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.221266 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.221267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.221268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.221469 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.221945 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.221994 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.222121 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260619 12:07:18.223654 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.214006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32928)
0619 12:07:18.214060 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.214072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.214125 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.214652 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.214653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.214791 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.214845 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.214847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.216178 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.216179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.216984 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.216987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.217831 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.217832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.217914 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.218320 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.218325 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.219882 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.222718 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.222719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.222720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.222893 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.223066 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.223066 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.223067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.223121 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.223325 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.223326 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.223399 (+ 73us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.223462 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.223561 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:07:18.223670 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.214228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32928 (local address 127.0.203.188:43609)
0619 12:07:18.214484 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.214486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.214496 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:18.214516 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.214516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.214527 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.214635 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.214943 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.215704 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.217381 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.217714 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.217814 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.219982 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.219983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.219984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.219991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.222630 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.222955 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.222956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.222956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.223013 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.223168 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.223168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.223169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.223275 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.223327 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.223482 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.223560 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":174,"thread_start_us":110,"threads_started":1}
I20260619 12:07:18.224949 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.217189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:33133 (local address 127.0.0.1:56830)
0619 12:07:18.217269 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.217283 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.217348 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.217478 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.217479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.217625 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.217688 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.217690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.218494 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.218495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.219184 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.219187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.219704 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.219705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.219785 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.220175 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.220179 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.221760 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.223987 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.223987 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.223988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.224160 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.224376 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.224377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.224377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.224430 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.224660 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.224661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.224700 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.224768 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.224860 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 12:07:18.224962 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.217289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.0.203.189:33133)
0619 12:07:18.217369 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.217371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.217380 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:07:18.217410 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.217411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.217420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.217469 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.217755 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.218395 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.219286 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.219589 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.219648 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.221865 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.221867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.221867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.221889 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.223921 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.224258 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.224259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.224260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.224318 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.224491 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.224491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.224492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.224602 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.224657 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.224798 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.224879 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:07:18.225997 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: Bootstrap starting.
I20260619 12:07:18.226895 14190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:18.227218 14190 log.cc:824] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:18.228008 14340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:07:18.228542 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: No bootstrap required, opened a new log
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:18.230638 14190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.230758 14190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:18.230780 14190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27cde3ba9db64f899efeae199e6aa393, State: Initialized, Role: FOLLOWER
I20260619 12:07:18.231148 14190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.231714 14190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:18.231657 14347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.231848 14347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
I20260619 12:07:18.233088 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.223517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.0.203.190:33075)
0619 12:07:18.223628 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.223630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.223704 (+ 74us) server_negotiation.cc:407] Connection header received
0619 12:07:18.223808 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.223809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.223822 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.223868 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.224674 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.225301 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.226227 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.226445 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.226516 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.229278 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.229280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.229280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.229289 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.230796 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.232090 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.232091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.232092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.232174 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.232390 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.232392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.232392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.232528 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.232653 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.232885 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.232994 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260619 12:07:18.234777 14358 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:07:18.234849 14358 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:07:18.235731 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.228420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.0.203.188:43609)
0619 12:07:18.228475 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.228476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.228486 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:18.228509 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.228510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.228520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.228586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.228871 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.229449 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.230424 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.230796 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.230873 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.232946 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.232948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.232948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.232956 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.234376 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.234747 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.234748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.234748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.234797 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.234952 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.234953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.234953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.235145 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.235218 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.235524 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.235635 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:07:18.235860 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.228301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32930)
0619 12:07:18.228385 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.228395 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.228430 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.228602 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.228602 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.228750 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.228804 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.228806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.229544 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.229545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.230310 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.230313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.230883 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.230884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.230946 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.231306 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.231311 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.232850 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.234469 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.234470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.234470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.234677 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.234855 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.234855 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.234856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.234904 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.235231 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.235232 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.235288 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.235488 (+ 200us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.235757 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":96}
I20260619 12:07:18.237232 14340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3: Bootstrap starting.
I20260619 12:07:18.237848 14340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:18.238056 14340 log.cc:824] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:18.235844 14122 master_runner.cc:186] Error getting master registration for 127.0.203.189:33133: OK, instance_id { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" instance_seqno: 1781870838031819 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:07:18.238255 14122 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:07:18.239027 14340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3: No bootstrap required, opened a new log
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:07:18.241273 14340 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.241454 14340 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:18.241492 14340 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a71fc0f012540ceb9c38b8eeb7801f3, State: Initialized, Role: FOLLOWER
I20260619 12:07:18.241632 14340 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.241986 14340 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:18.244030 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.234161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.0.203.189:33133)
0619 12:07:18.234208 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.234210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.234281 (+ 71us) server_negotiation.cc:407] Connection header received
0619 12:07:18.234388 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.234389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.234401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.234429 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.235042 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.235766 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.238813 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.239018 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.239079 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.241310 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.241312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.241312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.241320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.242882 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.243230 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.243232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.243232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.243296 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.243517 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.243518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.243519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.243659 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.243836 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.243871 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.243941 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 12:07:18.244784 14372 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:07:18.244845 14372 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:07:18.243844 14361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.245183 14361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:07:18.246141 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:07:18.246483 14272 master_runner.cc:186] Error getting master registration for 127.0.203.189:33133: OK, instance_id { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" instance_seqno: 1781870838031819 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:07:18.249490 14272 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:07:18.263890 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.247599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.0.203.189:33133)
0619 12:07:18.247646 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.247649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.248165 (+ 516us) server_negotiation.cc:407] Connection header received
0619 12:07:18.248224 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.248224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.248235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.248279 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.250091 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.250856 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.252181 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.252486 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.252549 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.259310 (+ 6761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.259312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.259312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.259322 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.262069 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.262795 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.262798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.262798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.262893 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.263116 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.263117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.263117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.263279 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.263396 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.263659 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.263766 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:07:18.264572 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.248030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.0.203.188:43609)
0619 12:07:18.248067 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.248069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.248308 (+ 239us) server_negotiation.cc:407] Connection header received
0619 12:07:18.248386 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.248387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.248397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.248423 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.249086 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.249839 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.250889 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.251218 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.251274 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.258588 (+ 7314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.258590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.258591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.258602 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.260577 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.262640 (+ 2063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.262642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.262642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.262714 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.263055 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.263056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.263056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.263226 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.263333 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.264375 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.264476 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:07:18.265359 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.247956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.0.203.190:33075)
0619 12:07:18.248002 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.248004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.248012 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:07:18.248036 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.248036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.248047 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.248092 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.250490 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.251222 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.252551 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.252881 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.252963 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.259723 (+ 6760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.259725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.259726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.259736 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.261343 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.263460 (+ 2117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.263461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.263462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.263540 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.263912 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.263913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.263914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.264058 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.264169 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.265175 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.265273 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:07:18.274941 14265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } attempt: 3
Jun 19 12:07:18 dist-test-slave-44vj krb5kdc[14109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
I20260619 12:07:18.284145 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.275325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:43609 (local address 127.0.0.1:32948)
0619 12:07:18.275408 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:18.275420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:18.275462 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:18.275675 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:18.275676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:18.275870 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:18.275931 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.275933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.276949 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.276950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.277833 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:18.277836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.278497 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.278498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:18.278566 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.279158 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:18.279164 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:18.281042 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:18.283030 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.283032 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.283032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.283266 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.283497 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:18.283498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:18.283498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:18.283553 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:18.283840 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:18.283841 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:18.283890 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 12:07:18.283942 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:07:18.284039 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 12:07:18.284806 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:18.275443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32948 (local address 127.0.203.188:43609)
0619 12:07:18.275514 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:18.275517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:18.275528 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:07:18.275560 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:18.275561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:18.275573 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:18.275655 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:18.276018 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.276857 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.278074 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:18.278391 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:18.278470 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:18.281172 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:18.281174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:18.281174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:18.281185 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:18.282905 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.283345 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.283346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.283346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.283429 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:18.283613 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:18.283614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:18.283614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:18.283756 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:18.283814 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:18.284630 (+ 816us) server_negotiation.cc:299] Negotiation successful
0619 12:07:18.284714 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:07:18.286881 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: Bootstrap starting.
I20260619 12:07:18.287740 14265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:18.288074 14265 log.cc:824] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:18.289212 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: No bootstrap required, opened a new log
I20260619 12:07:18.291267 14265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.291481 14265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:18.291515 14265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9e12f923084c0f8f63c1821c19795a, State: Initialized, Role: FOLLOWER
I20260619 12:07:18.291672 14265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.292022 14265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:18.293226 14361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:18.293368 14361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.293694 14361 leader_election.cc:288] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27cde3ba9db64f899efeae199e6aa393 (127.0.203.190:33075), 2b9e12f923084c0f8f63c1821c19795a (127.0.203.189:33133)
I20260619 12:07:18.294129 14164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27cde3ba9db64f899efeae199e6aa393" is_pre_election: true
I20260619 12:07:18.294296 14164 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a71fc0f012540ceb9c38b8eeb7801f3 in term 0.
I20260619 12:07:18.294752 14239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9e12f923084c0f8f63c1821c19795a" is_pre_election: true
I20260619 12:07:18.294889 14239 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a71fc0f012540ceb9c38b8eeb7801f3 in term 0.
W20260619 12:07:18.294996 14393 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:07:18.295068 14393 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:07:18.295095 14290 leader_election.cc:302] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27cde3ba9db64f899efeae199e6aa393, 7a71fc0f012540ceb9c38b8eeb7801f3; no voters:
I20260619 12:07:18.295624 14361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:07:18.295684 14361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:07:18.295707 14361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:18.296834 14361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.296959 14361 leader_election.cc:288] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [CANDIDATE]: Term 1 election: Requested vote from peers 27cde3ba9db64f899efeae199e6aa393 (127.0.203.190:33075), 2b9e12f923084c0f8f63c1821c19795a (127.0.203.189:33133)
I20260619 12:07:18.298050 14197 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260619 12:07:18.298156 14197 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:07:18.299280 14239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9e12f923084c0f8f63c1821c19795a"
I20260619 12:07:18.299374 14239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:18.300408 14239 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a71fc0f012540ceb9c38b8eeb7801f3 in term 1.
I20260619 12:07:18.300619 14291 leader_election.cc:302] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9e12f923084c0f8f63c1821c19795a, 7a71fc0f012540ceb9c38b8eeb7801f3; no voters:
I20260619 12:07:18.300825 14164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27cde3ba9db64f899efeae199e6aa393"
I20260619 12:07:18.300923 14164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:18.300689 14382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.301182 14382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:07:18.302331 14164 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a71fc0f012540ceb9c38b8eeb7801f3 in term 1.
I20260619 12:07:18.302745 14361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:18.302927 14361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [term 1 LEADER]: Becoming Leader. State: Replica: 7a71fc0f012540ceb9c38b8eeb7801f3, State: Running, Role: LEADER
I20260619 12:07:18.303033 14361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } }
I20260619 12:07:18.303519 14361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a71fc0f012540ceb9c38b8eeb7801f3. Latest consensus state: current_term: 1 leader_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.303588 14361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:18.303927 14397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:18.304188 14397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:07:18.305996 14239 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [term 1 FOLLOWER]: Refusing update from remote peer 7a71fc0f012540ceb9c38b8eeb7801f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:18.306399 14361 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:18.308907 14382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a71fc0f012540ceb9c38b8eeb7801f3. Latest consensus state: current_term: 1 leader_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.309078 14382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:07:18.309523 14164 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [term 1 FOLLOWER]: Refusing update from remote peer 7a71fc0f012540ceb9c38b8eeb7801f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:18.309827 14361 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:18.311697 14347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a71fc0f012540ceb9c38b8eeb7801f3. Latest consensus state: current_term: 1 leader_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.311863 14347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:07:18.313575 14395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.313669 14395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:18.314208 14347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.314330 14347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:07:18.314445 14398 mvcc.cc:205] Tried to move back new op lower bound from 7298542953699192832 to 7298542953689907200. Current Snapshot: MvccSnapshot[applied={T|T < 7298542953699192832}]
I20260619 12:07:18.314843 14401 mvcc.cc:205] Tried to move back new op lower bound from 7298542953699192832 to 7298542953689907200. Current Snapshot: MvccSnapshot[applied={T|T < 7298542953699192832}]
I20260619 12:07:18.315281 14361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.315379 14361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:18.316051 14382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27cde3ba9db64f899efeae199e6aa393" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33075 } } peers { permanent_uuid: "2b9e12f923084c0f8f63c1821c19795a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 33133 } } peers { permanent_uuid: "7a71fc0f012540ceb9c38b8eeb7801f3" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 43609 } } }
I20260619 12:07:18.316198 14382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:07:18.316392 14399 mvcc.cc:205] Tried to move back new op lower bound from 7298542953699192832 to 7298542953689907200. Current Snapshot: MvccSnapshot[applied={T|T < 7298542953699192832}]
I20260619 12:07:18.318667 14397 catalog_manager.cc:1347] Generated new cluster ID: 47b2406c6c694cb986a2167d97241aa1
I20260619 12:07:18.318712 14397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:07:18.358791 14397 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:07:18.359869 14397 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:07:18.379535 14397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a71fc0f012540ceb9c38b8eeb7801f3: Generated new TSK 0
I20260619 12:07:18.380028 14397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:07:19.235466 14358 catalog_manager.cc:1259] Loaded cluster ID: 47b2406c6c694cb986a2167d97241aa1
I20260619 12:07:19.235543 14358 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: loading cluster ID for follower catalog manager: success
I20260619 12:07:19.236402 14358 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: acquiring CA information for follower catalog manager: success
I20260619 12:07:19.236725 14358 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 27cde3ba9db64f899efeae199e6aa393: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:07:19.295696 14393 catalog_manager.cc:1259] Loaded cluster ID: 47b2406c6c694cb986a2167d97241aa1
I20260619 12:07:19.295810 14393 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: loading cluster ID for follower catalog manager: success
I20260619 12:07:19.296713 14393 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: acquiring CA information for follower catalog manager: success
I20260619 12:07:19.297024 14393 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b9e12f923084c0f8f63c1821c19795a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:07:19.465334 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14122
I20260619 12:07:19.470440 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14197
I20260619 12:07:19.475306 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14272
2026-06-19T12:07:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2853 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:07:19 dist-test-slave-44vj krb5kdc[14412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:07:19 dist-test-slave-44vj krb5kdc[14412](info): set up 2 sockets
Jun 19 12:07:19 dist-test-slave-44vj krb5kdc[14412](info): commencing operation
krb5kdc: starting...
W20260619 12:07:20.459591 814 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:07:20 dist-test-slave-44vj krb5kdc[14412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:07:20.584501 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34509
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:42105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:34509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:20.671824 14425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:20.672089 14425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:20.672120 14425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:20.673424 14425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:20.673471 14425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:20.673489 14425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:20.673506 14425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:20.673520 14425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:20.675146 14425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:34509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:20.675473 14425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:20.675719 14425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:20.681150 14431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:20.681139 14433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:20.681130 14430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:20.681668 14425 server_base.cc:1034] running on GCE node
I20260619 12:07:20.681954 14425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:20.682296 14425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:20.683436 14425 hybrid_clock.cc:648] HybridClock initialized: now 1781870840683386 us; error 72 us; skew 500 ppm
Jun 19 12:07:20 dist-test-slave-44vj krb5kdc[14412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:20.686496 14425 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:20.687067 14425 webserver.cc:460] Webserver started at http://127.0.203.190:42809/ using document root <none> and password file <none>
I20260619 12:07:20.687299 14425 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:20.687350 14425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:20.687526 14425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:20.688783 14425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "1193ad1423d147019f7b5e84ffa6874b"
format_stamp: "Formatted at 2026-06-19 12:07:20 on dist-test-slave-44vj"
server_key: "8525600963bafea57b5a541791e4e793"
server_key_iv: "23a81eea90adfb3ec082f94fa628cf34"
server_key_version: "encryptionkey@0"
I20260619 12:07:20.689172 14425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "1193ad1423d147019f7b5e84ffa6874b"
format_stamp: "Formatted at 2026-06-19 12:07:20 on dist-test-slave-44vj"
server_key: "8525600963bafea57b5a541791e4e793"
server_key_iv: "23a81eea90adfb3ec082f94fa628cf34"
server_key_version: "encryptionkey@0"
I20260619 12:07:20.690765 14425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:07:20.691848 14440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:20.692113 14425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:20.692221 14425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "1193ad1423d147019f7b5e84ffa6874b"
format_stamp: "Formatted at 2026-06-19 12:07:20 on dist-test-slave-44vj"
server_key: "8525600963bafea57b5a541791e4e793"
server_key_iv: "23a81eea90adfb3ec082f94fa628cf34"
server_key_version: "encryptionkey@0"
I20260619 12:07:20.692293 14425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:20.710440 14425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:20.712908 14425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:20.713070 14425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:20.717705 14425 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:34509
I20260619 12:07:20.717773 14492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:34509 every 8 connection(s)
I20260619 12:07:20.718133 14425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:20.719105 14493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:20.720518 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14425
I20260619 12:07:20.720630 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:07:20.720821 814 external_mini_cluster.cc:1428] Setting key af0f4a234990d48f51707e3dbbcecdb9
I20260619 12:07:20.722018 14493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b: Bootstrap starting.
I20260619 12:07:20.722772 14493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:20.723063 14493 log.cc:824] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:20.723966 14493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b: No bootstrap required, opened a new log
I20260619 12:07:20.726218 14493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1193ad1423d147019f7b5e84ffa6874b" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34509 } }
I20260619 12:07:20.726436 14493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:20.726480 14493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1193ad1423d147019f7b5e84ffa6874b, State: Initialized, Role: FOLLOWER
I20260619 12:07:20.726632 14493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1193ad1423d147019f7b5e84ffa6874b" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34509 } }
I20260619 12:07:20.726716 14493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:07:20.726748 14493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:07:20.726799 14493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:20.727500 14493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1193ad1423d147019f7b5e84ffa6874b" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34509 } }
I20260619 12:07:20.727605 14493 leader_election.cc:302] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1193ad1423d147019f7b5e84ffa6874b; no voters:
I20260619 12:07:20.727818 14493 leader_election.cc:288] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:07:20.727866 14498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:20.728044 14498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [term 1 LEADER]: Becoming Leader. State: Replica: 1193ad1423d147019f7b5e84ffa6874b, State: Running, Role: LEADER
I20260619 12:07:20.728116 14493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:20.728343 14498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1193ad1423d147019f7b5e84ffa6874b" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34509 } }
Jun 19 12:07:20 dist-test-slave-44vj krb5kdc[14412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:20.728756 14499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1193ad1423d147019f7b5e84ffa6874b. Latest consensus state: current_term: 1 leader_uuid: "1193ad1423d147019f7b5e84ffa6874b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1193ad1423d147019f7b5e84ffa6874b" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34509 } } }
I20260619 12:07:20.728857 14499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:20.729019 14500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1193ad1423d147019f7b5e84ffa6874b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1193ad1423d147019f7b5e84ffa6874b" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34509 } } }
I20260619 12:07:20.729074 14500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:20.729202 14505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:20.730069 14505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:07:20.732420 14505 catalog_manager.cc:1347] Generated new cluster ID: 010a9b88a64443bc853a9ab210ad293b
I20260619 12:07:20.732510 14505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:07:20.733232 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:20.721455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.0.203.190:34509)
0619 12:07:20.721734 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:20.721744 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:20.721782 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:07:20.722349 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:20.722359 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:20.722430 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:20.722605 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:20.723296 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:20.724629 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:20.725519 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:20.726203 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:20.726344 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:20.729225 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:20.729232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:20.729232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:20.729248 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:20.731409 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:20.731818 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:20.731822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:20.731823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:20.731951 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:20.732156 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:20.732157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:20.732157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:20.732484 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:20.732625 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:20.732856 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:07:20.733015 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":141,"threads_started":1}
I20260619 12:07:20.734311 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:07:20.750217 14505 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:07:20.750943 14505 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:07:20.756686 14505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1193ad1423d147019f7b5e84ffa6874b: Generated new TSK 0
I20260619 12:07:20.756958 14505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:07:20.788847 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14425
2026-06-19T12:07:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1311 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:07:20 dist-test-slave-44vj krb5kdc[14520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:07:20 dist-test-slave-44vj krb5kdc[14520](info): set up 2 sockets
Jun 19 12:07:20 dist-test-slave-44vj krb5kdc[14520](info): commencing operation
krb5kdc: starting...
W20260619 12:07:21.751551 814 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:07:21 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:07:21.878826 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38787
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:38787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:21.974280 14533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:21.974614 14533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:21.974650 14533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:21.975970 14533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:21.976019 14533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:21.976037 14533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:21.976054 14533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:21.976068 14533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:21.977721 14533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:38787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:21.978030 14533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:21.978282 14533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:21.983889 14539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:21.983958 14541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:21.983892 14538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:21.984107 14533 server_base.cc:1034] running on GCE node
I20260619 12:07:21.984565 14533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:21.984930 14533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:21.986099 14533 hybrid_clock.cc:648] HybridClock initialized: now 1781870841986060 us; error 58 us; skew 500 ppm
Jun 19 12:07:21 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:21.989225 14533 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:21.989861 14533 webserver.cc:460] Webserver started at http://127.0.203.190:33667/ using document root <none> and password file <none>
I20260619 12:07:21.990103 14533 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:21.990152 14533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:21.990298 14533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:21.991588 14533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "47e81fc6b76f4319b71b47fba0121014"
format_stamp: "Formatted at 2026-06-19 12:07:21 on dist-test-slave-44vj"
server_key: "3caabd9067aedc837fe3b150b30f45bf"
server_key_iv: "c9ae127aa0a1af43b26f574bd53d1b9d"
server_key_version: "encryptionkey@0"
I20260619 12:07:21.991981 14533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "47e81fc6b76f4319b71b47fba0121014"
format_stamp: "Formatted at 2026-06-19 12:07:21 on dist-test-slave-44vj"
server_key: "3caabd9067aedc837fe3b150b30f45bf"
server_key_iv: "c9ae127aa0a1af43b26f574bd53d1b9d"
server_key_version: "encryptionkey@0"
I20260619 12:07:21.993490 14533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:21.994365 14548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:21.994598 14533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:21.994693 14533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "47e81fc6b76f4319b71b47fba0121014"
format_stamp: "Formatted at 2026-06-19 12:07:21 on dist-test-slave-44vj"
server_key: "3caabd9067aedc837fe3b150b30f45bf"
server_key_iv: "c9ae127aa0a1af43b26f574bd53d1b9d"
server_key_version: "encryptionkey@0"
I20260619 12:07:21.994771 14533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:22.026499 14533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:22.029121 14533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:22.029287 14533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:22.034380 14533 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:38787
I20260619 12:07:22.034461 14600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:38787 every 8 connection(s)
I20260619 12:07:22.034868 14533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:22.035662 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14533
I20260619 12:07:22.035799 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:07:22.036015 814 external_mini_cluster.cc:1428] Setting key 168097ba4d84f6a955c99b7a99256f95
I20260619 12:07:22.036270 14601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:22.040050 14601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Bootstrap starting.
I20260619 12:07:22.040764 14601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:22.041047 14601 log.cc:824] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:22.041782 14601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: No bootstrap required, opened a new log
Jun 19 12:07:22 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:22.044299 14601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:22.044529 14601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:22.044556 14601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e81fc6b76f4319b71b47fba0121014, State: Initialized, Role: FOLLOWER
I20260619 12:07:22.044713 14601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:22.044799 14601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:07:22.044842 14601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:07:22.044904 14601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:22.045789 14601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:22.045918 14601 leader_election.cc:302] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47e81fc6b76f4319b71b47fba0121014; no voters:
I20260619 12:07:22.046144 14601 leader_election.cc:288] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:07:22.046193 14606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:22.046344 14606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 LEADER]: Becoming Leader. State: Replica: 47e81fc6b76f4319b71b47fba0121014, State: Running, Role: LEADER
I20260619 12:07:22.046515 14601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:22.046520 14606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:22.047123 14608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47e81fc6b76f4319b71b47fba0121014. Latest consensus state: current_term: 1 leader_uuid: "47e81fc6b76f4319b71b47fba0121014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } } }
I20260619 12:07:22.047402 14608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:22.047292 14607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47e81fc6b76f4319b71b47fba0121014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } } }
I20260619 12:07:22.047722 14607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:22.047905 14614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:22.048924 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.036783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42600 (local address 127.0.203.190:38787)
0619 12:07:22.037056 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.037067 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.037100 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:07:22.037962 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.037971 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.038033 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.038193 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:22.038937 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.040036 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.040860 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.041364 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.041481 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.044427 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:22.044432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:22.044432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:22.044448 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:22.046413 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.046867 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.046871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.046872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.046993 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.047212 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.047213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.047213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.047646 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:22.047819 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.048483 (+ 664us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.048666 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":130,"threads_started":1}
I20260619 12:07:22.049067 14614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:07:22.051469 14614 catalog_manager.cc:1347] Generated new cluster ID: 31f65254bd2b48a88a08127dac1d8212
I20260619 12:07:22.051584 14614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:07:22.085537 14614 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:07:22.086354 14614 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:07:22.100402 14614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Generated new TSK 0
I20260619 12:07:22.100699 14614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:07:22.134301 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38787
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:22.224165 14627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:22.224424 14627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:22.224462 14627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:22.225754 14627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:22.225801 14627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:22.225838 14627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:07:22.227355 14627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:22.227662 14627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:22.227942 14627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:22.233614 14635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:22.233642 14632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:22.233615 14633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:22.233898 14627 server_base.cc:1034] running on GCE node
I20260619 12:07:22.234163 14627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:22.234485 14627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:22.235668 14627 hybrid_clock.cc:648] HybridClock initialized: now 1781870842235637 us; error 50 us; skew 500 ppm
Jun 19 12:07:22 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:22.238845 14627 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:07:22.239528 14627 webserver.cc:460] Webserver started at http://127.0.203.129:39245/ using document root <none> and password file <none>
I20260619 12:07:22.239773 14627 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:22.239823 14627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:22.239985 14627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:22.241240 14627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "7ac1d4819ccf4ac4876fd2161995e41e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "de10647c5f7ee6c54bc49b304cbea5a1"
server_key_iv: "d95caa5b1af0e7f4b4a6214452d43c08"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.241628 14627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "7ac1d4819ccf4ac4876fd2161995e41e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "de10647c5f7ee6c54bc49b304cbea5a1"
server_key_iv: "d95caa5b1af0e7f4b4a6214452d43c08"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.243175 14627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:07:22.244180 14642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.244498 14627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:22.244598 14627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "7ac1d4819ccf4ac4876fd2161995e41e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "de10647c5f7ee6c54bc49b304cbea5a1"
server_key_iv: "d95caa5b1af0e7f4b4a6214452d43c08"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.244674 14627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:22.279553 14627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:22.282320 14627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:22.282474 14627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:22.282734 14627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:22.283104 14627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:22.283139 14627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.283171 14627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:22.283195 14627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.290374 14627 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:35665
I20260619 12:07:22.290450 14755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:35665 every 8 connection(s)
I20260619 12:07:22.290832 14627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:07:22.291057 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14627
I20260619 12:07:22.291175 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:07:22.291419 814 external_mini_cluster.cc:1428] Setting key f43a4e567554ccef61eeb11a66948f8b
Jun 19 12:07:22 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
I20260619 12:07:22.302362 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.291857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:57083 (local address 127.0.203.190:38787)
0619 12:07:22.291920 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.291922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.292273 (+ 351us) server_negotiation.cc:407] Connection header received
0619 12:07:22.292999 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.293000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.293013 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.293095 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:22.294397 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.295187 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.296373 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.296607 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.296676 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.299316 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:22.299317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:22.299318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:22.299326 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:22.300881 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.301338 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.301340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.301340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.301398 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.301625 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.301626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.301627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.301755 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:22.301813 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.302161 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.302270 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:07:22.302438 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.291870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:57083)
0619 12:07:22.292102 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.292167 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.292843 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.293188 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.293193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.293994 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:22.294284 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.294289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.295309 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.295310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.296251 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.296254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.296694 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.296695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.296837 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.297620 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:22.297631 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:22.299216 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:22.301029 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:22.301038 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:22.301041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:22.301266 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:22.301454 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:22.301455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:22.301455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:22.301570 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:22.301812 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:22.301818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:22.301972 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.302094 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.302240 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":124,"threads_started":1}
I20260619 12:07:22.303115 14756 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38787
I20260619 12:07:22.303220 14756 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:22.303472 14756 heartbeater.cc:511] Master 127.0.203.190:38787 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
I20260619 12:07:22.304124 14565 ts_manager.cc:194] Registered new tserver with Master: 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665)
I20260619 12:07:22.304734 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:57083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:07:22.320040 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38787
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:22.407732 14761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:22.407984 14761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:22.408015 14761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:22.409330 14761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:22.409387 14761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:22.409418 14761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:07:22.411078 14761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:22.411324 14761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:22.411566 14761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:22.417321 14766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:22.417346 14767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:22.417505 14761 server_base.cc:1034] running on GCE node
W20260619 12:07:22.417321 14769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:22.417846 14761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:22.418110 14761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:22.419308 14761 hybrid_clock.cc:648] HybridClock initialized: now 1781870842419284 us; error 36 us; skew 500 ppm
Jun 19 12:07:22 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:22.422561 14761 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:07:22.423117 14761 webserver.cc:460] Webserver started at http://127.0.203.130:37273/ using document root <none> and password file <none>
I20260619 12:07:22.423421 14761 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:22.423478 14761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:22.423606 14761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:22.424741 14761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "47b1d7ead78a410291a57e2b926e201e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "ee09afb10915f126588fc9e7ac1c7b6e"
server_key_iv: "3af33e3840b21a309d6ef8760691597b"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.425094 14761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "47b1d7ead78a410291a57e2b926e201e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "ee09afb10915f126588fc9e7ac1c7b6e"
server_key_iv: "3af33e3840b21a309d6ef8760691597b"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.426759 14761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:07:22.427850 14776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.428211 14761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:22.428342 14761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "47b1d7ead78a410291a57e2b926e201e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "ee09afb10915f126588fc9e7ac1c7b6e"
server_key_iv: "3af33e3840b21a309d6ef8760691597b"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.428429 14761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:22.438345 14761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:22.440904 14761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:22.441058 14761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:22.441752 14761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:22.442435 14761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:22.442512 14761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.442546 14761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:22.442564 14761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.449776 14761 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:33671
I20260619 12:07:22.449951 14889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:33671 every 8 connection(s)
I20260619 12:07:22.450229 14761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:07:22.456113 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14761
I20260619 12:07:22.456243 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:07:22.456439 814 external_mini_cluster.cc:1428] Setting key c423859b233fdb0c72a5e3cd86365144
Jun 19 12:07:22 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:22.462426 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.451236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:33237 (local address 127.0.203.190:38787)
0619 12:07:22.451325 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.451328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.451642 (+ 314us) server_negotiation.cc:407] Connection header received
0619 12:07:22.452218 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.452220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.452235 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.452317 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:22.453408 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.454008 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.455250 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.455548 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.455622 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.458600 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:22.458602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:22.458602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:22.458612 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:22.460226 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.460973 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.460975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.460975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.461128 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.461528 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.461530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.461531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.461701 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:22.461775 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.462241 (+ 466us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.462346 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:07:22.462759 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.451168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:33237)
0619 12:07:22.451416 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.451482 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.452061 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.452407 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.452411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.453043 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:22.453265 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.453271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.454183 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.454185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.455117 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.455120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.455668 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.455670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.455806 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.456613 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:22.456623 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:22.458483 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:22.460374 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:22.460391 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:22.460395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:22.460813 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:22.461230 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:22.461231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:22.461232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:22.461416 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:22.461784 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:22.461791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:22.462043 (+ 252us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.462199 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.462445 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260619 12:07:22.463475 14890 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38787
I20260619 12:07:22.463598 14890 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:22.463814 14890 heartbeater.cc:511] Master 127.0.203.190:38787 requested a full tablet report, sending...
I20260619 12:07:22.464278 14565 ts_manager.cc:194] Registered new tserver with Master: 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671)
I20260619 12:07:22.464877 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:33237
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:07:22.492072 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38787
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:22.584930 14895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:22.585210 14895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:22.585239 14895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:22.586652 14895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:22.586709 14895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:22.586740 14895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:07:22.588421 14895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:22.588723 14895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:22.588984 14895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:22.594604 14900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:22.594620 14901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:22.594857 14895 server_base.cc:1034] running on GCE node
W20260619 12:07:22.594596 14903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:22.595109 14895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:22.595335 14895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:22.596539 14895 hybrid_clock.cc:648] HybridClock initialized: now 1781870842596514 us; error 44 us; skew 500 ppm
Jun 19 12:07:22 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:22.599561 14895 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:07:22.600183 14895 webserver.cc:460] Webserver started at http://127.0.203.131:44643/ using document root <none> and password file <none>
I20260619 12:07:22.600503 14895 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:22.600579 14895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:22.600694 14895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:22.601940 14895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "a1cd4b9cfbb144ce8180495c8e650bc0"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "0ffe0bedb01da35d3ce77420e29e8c92"
server_key_iv: "377dce400f62642dfc0fe03ab31dd8b5"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.602303 14895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "a1cd4b9cfbb144ce8180495c8e650bc0"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "0ffe0bedb01da35d3ce77420e29e8c92"
server_key_iv: "377dce400f62642dfc0fe03ab31dd8b5"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.603821 14895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:07:22.604928 14910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.605227 14895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:22.605340 14895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "a1cd4b9cfbb144ce8180495c8e650bc0"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "0ffe0bedb01da35d3ce77420e29e8c92"
server_key_iv: "377dce400f62642dfc0fe03ab31dd8b5"
server_key_version: "encryptionkey@0"
I20260619 12:07:22.605429 14895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:22.618680 14895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:22.621210 14895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:22.621429 14895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:22.621694 14895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:22.622025 14895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:22.622058 14895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.622090 14895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:22.622114 14895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:22.629166 14895 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:46719
I20260619 12:07:22.629247 15023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:46719 every 8 connection(s)
I20260619 12:07:22.629624 14895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:07:22 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:22.638677 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 14895
I20260619 12:07:22.638805 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:07:22.638983 814 external_mini_cluster.cc:1428] Setting key 25d421c79a37897716cd5e0ac8b4a6b8
I20260619 12:07:22.640929 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.630603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:38697 (local address 127.0.203.190:38787)
0619 12:07:22.630717 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.630719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.631022 (+ 303us) server_negotiation.cc:407] Connection header received
0619 12:07:22.631671 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.631673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.631685 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.631750 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:22.632841 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.633407 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.634566 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.634772 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.634832 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.637518 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:22.637520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:22.637521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:22.637530 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:22.639047 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.639689 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.639691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.639691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.639800 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.640206 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.640207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.640208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.640377 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:22.640452 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.640736 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.640825 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 12:07:22.641059 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.630503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:38697)
0619 12:07:22.630817 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.630866 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.631541 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.631827 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.631831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.632451 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:22.632737 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.632742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.633532 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.633533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.634461 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.634463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.634883 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.634884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.635006 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.635648 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:22.635657 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:22.637316 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:22.639202 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:22.639216 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:22.639219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:22.639608 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:22.639894 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:22.639895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:22.639896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:22.640077 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:22.640448 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:22.640453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:22.640598 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.640698 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.640855 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":102,"threads_started":1}
I20260619 12:07:22.641624 15024 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38787
I20260619 12:07:22.641726 15024 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:22.641903 15024 heartbeater.cc:511] Master 127.0.203.190:38787 requested a full tablet report, sending...
I20260619 12:07:22.642253 14565 ts_manager.cc:194] Registered new tserver with Master: a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719)
I20260619 12:07:22.642655 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:38697
I20260619 12:07:22.651661 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:07:22.659972 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.653373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.0.203.190:38787)
0619 12:07:22.653436 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.653438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.653452 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:07:22.653527 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.653528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.653539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.653592 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:22.654219 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.654765 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.655552 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.655752 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.655804 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.657042 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:22.657044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:22.657044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:22.657051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:22.658710 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.659088 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.659090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.659090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.659151 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:22.659312 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:22.659313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:22.659313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:22.659552 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:22.659650 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.659759 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.659857 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:07:22.662360 14565 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:07:22.664227 14565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:07:22.672144 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.667317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35665 (local address 127.0.0.1:45514)
0619 12:07:22.667567 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.667611 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.667693 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.668032 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.668036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.668038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:22.668167 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.668171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.669717 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.669719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.670803 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.670806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.671797 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.671798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.671886 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.671901 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.671953 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.672021 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":75,"thread_start_us":156,"threads_started":1}
I20260619 12:07:22.672210 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.667722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.0.203.130:33671)
0619 12:07:22.668096 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.668102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.668112 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:22.668158 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.668160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.668177 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.668399 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:22.668471 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.669586 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.671068 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.671793 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.671865 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.671940 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.672073 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.672127 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":111,"threads_started":1}
I20260619 12:07:22.672303 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.667331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:33671 (local address 127.0.0.1:58842)
0619 12:07:22.667916 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.667925 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.667966 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.668278 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.668279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.668281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:22.668331 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.668333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.669699 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.669700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.670969 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.670972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.671893 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.671894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.671967 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.671976 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.672052 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.672118 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":72,"thread_start_us":167,"threads_started":1}
I20260619 12:07:22.672303 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.667317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:46719 (local address 127.0.0.1:60584)
0619 12:07:22.667806 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.667815 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.667856 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.668362 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.668364 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.668367 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:22.668468 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.668471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.669549 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.669550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.670968 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.670972 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.672090 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.672091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.672143 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.672146 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.672173 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.672217 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":4,"thread_start_us":82,"threads_started":1}
I20260619 12:07:22.672883 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.667532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.0.203.129:35665)
0619 12:07:22.667828 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.667834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.667850 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:07:22.667896 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.667899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.667916 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.668025 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:22.668281 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.669575 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.670888 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.671688 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.672432 (+ 744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.672700 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.672733 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.672789 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":97,"threads_started":1}
I20260619 12:07:22.673230 14824 tablet_service.cc:1467] Processing CreateTablet for tablet f17440a91deb47b789b2412b2841ae06 (DEFAULT_TABLE table=test-table [id=c0a589590e9a4de5ad3e97cccd06af9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:22.673607 14824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f17440a91deb47b789b2412b2841ae06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:22.673543 14690 tablet_service.cc:1467] Processing CreateTablet for tablet f17440a91deb47b789b2412b2841ae06 (DEFAULT_TABLE table=test-table [id=c0a589590e9a4de5ad3e97cccd06af9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:22.673861 14690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f17440a91deb47b789b2412b2841ae06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:22.675581 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.667762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.0.203.131:46719)
0619 12:07:22.668177 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.668181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.668192 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:07:22.668229 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.668231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.668245 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.668355 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:22.668689 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.669444 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.671436 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.672003 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.672341 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.675320 (+ 2979us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.675388 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.675471 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":113,"threads_started":1}
I20260619 12:07:22.676393 14958 tablet_service.cc:1467] Processing CreateTablet for tablet f17440a91deb47b789b2412b2841ae06 (DEFAULT_TABLE table=test-table [id=c0a589590e9a4de5ad3e97cccd06af9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:22.676780 14958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f17440a91deb47b789b2412b2841ae06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:22.676923 15041 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Bootstrap starting.
I20260619 12:07:22.677356 15042 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap starting.
I20260619 12:07:22.677806 15041 tablet_bootstrap.cc:654] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:22.678009 15042 tablet_bootstrap.cc:654] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:22.678498 15041 log.cc:824] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:22.678555 15042 log.cc:824] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:22.679493 15041 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: No bootstrap required, opened a new log
I20260619 12:07:22.679570 15041 ts_tablet_manager.cc:1397] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:22.679612 15042 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: No bootstrap required, opened a new log
I20260619 12:07:22.679699 15042 ts_tablet_manager.cc:1397] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:07:22.681278 15042 raft_consensus.cc:348] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.681309 15041 raft_consensus.cc:348] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.681447 15042 raft_consensus.cc:374] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:22.681473 15042 raft_consensus.cc:729] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac1d4819ccf4ac4876fd2161995e41e, State: Initialized, Role: FOLLOWER
I20260619 12:07:22.681476 15041 raft_consensus.cc:374] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:22.681512 15041 raft_consensus.cc:729] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b1d7ead78a410291a57e2b926e201e, State: Initialized, Role: FOLLOWER
I20260619 12:07:22.681594 15042 consensus_queue.cc:260] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.681754 15041 consensus_queue.cc:260] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.681788 15045 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Bootstrap starting.
I20260619 12:07:22.681926 14756 heartbeater.cc:503] Master 127.0.203.190:38787 was elected leader, sending a full tablet report...
I20260619 12:07:22.682101 15041 ts_tablet_manager.cc:1428] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:07:22.682178 15042 ts_tablet_manager.cc:1428] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:07:22.682240 14890 heartbeater.cc:503] Master 127.0.203.190:38787 was elected leader, sending a full tablet report...
I20260619 12:07:22.682945 15045 tablet_bootstrap.cc:654] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:22.683622 15045 log.cc:824] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:22.684407 15045 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: No bootstrap required, opened a new log
I20260619 12:07:22.684474 15045 ts_tablet_manager.cc:1397] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:22.685983 15045 raft_consensus.cc:348] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.686146 15045 raft_consensus.cc:374] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:22.686169 15045 raft_consensus.cc:729] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cd4b9cfbb144ce8180495c8e650bc0, State: Initialized, Role: FOLLOWER
I20260619 12:07:22.686272 15045 consensus_queue.cc:260] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.686537 15045 ts_tablet_manager.cc:1428] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:07:22.686666 15024 heartbeater.cc:503] Master 127.0.203.190:38787 was elected leader, sending a full tablet report...
W20260619 12:07:22.700830 14891 tablet.cc:2369] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:07:22.791738 14757 tablet.cc:2369] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:07:22.851423 15049 raft_consensus.cc:482] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:22.851567 15049 raft_consensus.cc:504] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.852002 15049 leader_election.cc:288] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665)
I20260619 12:07:22.856547 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.852232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35665 (local address 127.0.203.131:32803)
0619 12:07:22.852291 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.852327 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.852393 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.852728 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.852730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.852733 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:22.852835 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.852838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.853894 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.853895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.855306 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.855311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.856197 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.856198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.856304 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.856314 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.856378 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.856434 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":49}
I20260619 12:07:22.856557 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.852256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:33671 (local address 127.0.203.131:52269)
0619 12:07:22.852474 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.852485 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:22.852532 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:22.852749 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:22.852750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:22.852751 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:22.852835 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.852838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.853906 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.853907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.855310 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:22.855311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.856197 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.856198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:22.856304 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.856314 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 12:07:22.856378 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:07:22.856434 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":67,"thread_start_us":131,"threads_started":1}
I20260619 12:07:22.856598 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.852452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:32803 (local address 127.0.203.129:35665)
0619 12:07:22.852525 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.852528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.852542 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:07:22.852582 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.852584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.852596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.852669 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:22.852955 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.853769 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.855468 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.856107 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.856192 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.856204 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.856431 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.856498 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:07:22.856598 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.852489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:52269 (local address 127.0.203.130:33671)
0619 12:07:22.852565 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.852568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.852578 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:22.852608 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.852609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.852619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.852677 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:22.852936 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.853799 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.855479 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.856106 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.856192 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.856204 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.856454 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.856511 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:07:22.857061 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17440a91deb47b789b2412b2841ae06" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" is_pre_election: true
I20260619 12:07:22.857105 14844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17440a91deb47b789b2412b2841ae06" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b1d7ead78a410291a57e2b926e201e" is_pre_election: true
I20260619 12:07:22.857285 14844 raft_consensus.cc:2454] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 0.
I20260619 12:07:22.857270 14710 raft_consensus.cc:2454] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 0.
I20260619 12:07:22.857581 14913 leader_election.cc:302] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 47b1d7ead78a410291a57e2b926e201e, 7ac1d4819ccf4ac4876fd2161995e41e, a1cd4b9cfbb144ce8180495c8e650bc0; no voters:
I20260619 12:07:22.857757 15049 raft_consensus.cc:2781] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:07:22.857832 15049 raft_consensus.cc:482] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:07:22.857849 15049 raft_consensus.cc:3037] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:22.858546 15049 raft_consensus.cc:504] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.858777 15049 leader_election.cc:288] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 election: Requested vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665)
I20260619 12:07:22.858935 14844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17440a91deb47b789b2412b2841ae06" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b1d7ead78a410291a57e2b926e201e"
I20260619 12:07:22.859012 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17440a91deb47b789b2412b2841ae06" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac1d4819ccf4ac4876fd2161995e41e"
I20260619 12:07:22.859053 14844 raft_consensus.cc:3037] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:22.859103 14710 raft_consensus.cc:3037] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:22.860067 14844 raft_consensus.cc:2454] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 1.
I20260619 12:07:22.860067 14710 raft_consensus.cc:2454] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 1.
I20260619 12:07:22.860351 14913 leader_election.cc:302] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0; no voters:
I20260619 12:07:22.860469 15049 raft_consensus.cc:2781] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:22.860620 15049 raft_consensus.cc:686] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 LEADER]: Becoming Leader. State: Replica: a1cd4b9cfbb144ce8180495c8e650bc0, State: Running, Role: LEADER
I20260619 12:07:22.860716 15049 consensus_queue.cc:237] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:22.861812 14565 catalog_manager.cc:5495] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } health_report { overall_health: UNKNOWN } } }
W20260619 12:07:22.880347 15025 tablet.cc:2369] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:07:22.900385 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.897792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.0.203.131:46719)
0619 12:07:22.897912 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:22.897915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:22.897924 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:07:22.897973 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:22.897974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:22.897996 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:22.898075 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:07:22.898220 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.898798 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.899604 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:22.899806 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:22.899871 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:22.900000 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:07:22.900130 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:07:22.900209 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:22.900226 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 12:07:22.900250 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:07:22.902729 14844 raft_consensus.cc:1261] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Refusing update from remote peer a1cd4b9cfbb144ce8180495c8e650bc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:22.902815 14710 raft_consensus.cc:1261] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Refusing update from remote peer a1cd4b9cfbb144ce8180495c8e650bc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:22.903132 15051 consensus_queue.cc:1035] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:22.903307 15049 consensus_queue.cc:1035] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:22.906242 15054 mvcc.cc:205] Tried to move back new op lower bound from 7298542972526166016 to 7298542972358262784. Current Snapshot: MvccSnapshot[applied={T|T < 7298542972526166016}]
I20260619 12:07:22.907193 15057 mvcc.cc:205] Tried to move back new op lower bound from 7298542972526166016 to 7298542972358262784. Current Snapshot: MvccSnapshot[applied={T|T < 7298542972526166016 or (T in {7298542972526166016})}]
I20260619 12:07:22.907508 15055 mvcc.cc:205] Tried to move back new op lower bound from 7298542972526166016 to 7298542972358262784. Current Snapshot: MvccSnapshot[applied={T|T < 7298542972526166016 or (T in {7298542972526166016})}]
I20260619 12:07:22.910089 14564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:22.910221 14564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:22.911139 14564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Sending DeleteTablet for 3 replicas of tablet f17440a91deb47b789b2412b2841ae06
I20260619 12:07:22.911556 14824 tablet_service.cc:1514] Processing DeleteTablet for tablet f17440a91deb47b789b2412b2841ae06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:22 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:58842
I20260619 12:07:22.911535 14958 tablet_service.cc:1514] Processing DeleteTablet for tablet f17440a91deb47b789b2412b2841ae06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:22 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:60584
I20260619 12:07:22.911630 14690 tablet_service.cc:1514] Processing DeleteTablet for tablet f17440a91deb47b789b2412b2841ae06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:22 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:45514
I20260619 12:07:22.911929 15064 tablet_replica.cc:330] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: stopping tablet replica
I20260619 12:07:22.911934 15065 tablet_replica.cc:330] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: stopping tablet replica
I20260619 12:07:22.911947 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14627
I20260619 12:07:22.912056 15064 raft_consensus.cc:2229] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:07:22.912056 15065 raft_consensus.cc:2229] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:07:22.912181 15065 raft_consensus.cc:2258] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:22.912181 15064 raft_consensus.cc:2258] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:22.912387 15066 tablet_replica.cc:330] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: stopping tablet replica
I20260619 12:07:22.912489 15066 raft_consensus.cc:2229] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:07:22.912611 15066 raft_consensus.cc:2258] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:22.913084 15066 ts_tablet_manager.cc:1905] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:07:22.914449 15064 ts_tablet_manager.cc:1905] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:07:22.914566 15066 ts_tablet_manager.cc:1918] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:22.914626 15066 log.cc:1196] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/f17440a91deb47b789b2412b2841ae06
I20260619 12:07:22.914882 15066 ts_tablet_manager.cc:1939] T f17440a91deb47b789b2412b2841ae06 P a1cd4b9cfbb144ce8180495c8e650bc0: Deleting consensus metadata
I20260619 12:07:22.915401 14549 catalog_manager.cc:4841] TS a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719): tablet f17440a91deb47b789b2412b2841ae06 (table test-table [id=c0a589590e9a4de5ad3e97cccd06af9c]) successfully deleted
I20260619 12:07:22.915942 15064 ts_tablet_manager.cc:1918] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:22.915992 15064 log.cc:1196] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/f17440a91deb47b789b2412b2841ae06
I20260619 12:07:22.916167 15064 ts_tablet_manager.cc:1939] T f17440a91deb47b789b2412b2841ae06 P 47b1d7ead78a410291a57e2b926e201e: Deleting consensus metadata
I20260619 12:07:22.916610 14551 catalog_manager.cc:4841] TS 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671): tablet f17440a91deb47b789b2412b2841ae06 (table test-table [id=c0a589590e9a4de5ad3e97cccd06af9c]) successfully deleted
W20260619 12:07:22.919011 14552 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:35665 (error 108)
I20260619 12:07:22.920313 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:22.920177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35665 (local address 127.0.0.1:45516)
0619 12:07:22.920240 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:22.920279 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:07:22.920508 14552 catalog_manager.cc:4584] TS 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f17440a91deb47b789b2412b2841ae06: Network error: Client connection negotiation failed: client connection to 127.0.203.129:35665: connect: Connection refused (error 111)
I20260619 12:07:22.920641 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14761
I20260619 12:07:22.926748 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14895
I20260619 12:07:22.932318 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 14533
I20260619 12:07:22.938179 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38787
--webserver_interface=127.0.203.190
--webserver_port=33667
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:38787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:23.032570 15068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:23.032856 15068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:23.032883 15068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:23.034266 15068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:23.034327 15068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:23.034341 15068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:23.034354 15068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:23.034365 15068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:23.036160 15068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:38787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=33667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:23.036458 15068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:23.036738 15068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:23.047088 15077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.047106 15075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.047088 15074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:23.047478 15068 server_base.cc:1034] running on GCE node
I20260619 12:07:23.047873 15068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:23.048270 15068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:23.049455 15068 hybrid_clock.cc:648] HybridClock initialized: now 1781870843049419 us; error 64 us; skew 500 ppm
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:23.052377 15068 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:23.052906 15068 webserver.cc:460] Webserver started at http://127.0.203.190:33667/ using document root <none> and password file <none>
I20260619 12:07:23.053194 15068 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:23.053273 15068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:23.054869 15068 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:07:23.055660 15084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.055859 15068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:23.055958 15068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "47e81fc6b76f4319b71b47fba0121014"
format_stamp: "Formatted at 2026-06-19 12:07:21 on dist-test-slave-44vj"
server_key: "3caabd9067aedc837fe3b150b30f45bf"
server_key_iv: "c9ae127aa0a1af43b26f574bd53d1b9d"
server_key_version: "encryptionkey@0"
I20260619 12:07:23.056409 15068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:23.073966 15068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:23.076949 15068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:23.077158 15068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:23.081773 15068 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:38787
I20260619 12:07:23.081831 15136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:38787 every 8 connection(s)
I20260619 12:07:23.082249 15068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:23.084319 15137 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:07:23.084972 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15068
I20260619 12:07:23.085143 15137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Bootstrap starting.
I20260619 12:07:23.085289 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:35665
--local_ip_for_outbound_sockets=127.0.203.129
--tserver_master_addrs=127.0.203.190:38787
--webserver_port=39245
--webserver_interface=127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:23.096951 15137 log.cc:824] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:23.099344 15137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:07:23.099923 15137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Bootstrap complete.
I20260619 12:07:23.103238 15137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.103649 15137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e81fc6b76f4319b71b47fba0121014, State: Initialized, Role: FOLLOWER
I20260619 12:07:23.103844 15137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.103927 15137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:07:23.103989 15137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:07:23.104043 15137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:07:23.105448 15137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.105582 15137 leader_election.cc:302] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47e81fc6b76f4319b71b47fba0121014; no voters:
I20260619 12:07:23.105799 15137 leader_election.cc:288] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:07:23.105939 15141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:07:23.106145 15137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:23.106331 15141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 LEADER]: Becoming Leader. State: Replica: 47e81fc6b76f4319b71b47fba0121014, State: Running, Role: LEADER
I20260619 12:07:23.106487 15141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.106935 15142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "47e81fc6b76f4319b71b47fba0121014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } } }
I20260619 12:07:23.106942 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47e81fc6b76f4319b71b47fba0121014. Latest consensus state: current_term: 2 leader_uuid: "47e81fc6b76f4319b71b47fba0121014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } } }
I20260619 12:07:23.107059 15142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:23.107064 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:23.107801 15150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:23.108661 15150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:07:23.108764 15150 catalog_manager.cc:1259] Loaded cluster ID: 31f65254bd2b48a88a08127dac1d8212
I20260619 12:07:23.108796 15150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:07:23.109030 15150 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 12:07:23.111833 15150 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 12:07:23.114185 15150 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:07:23.114455 15150 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Loaded TSK: 0
I20260619 12:07:23.114693 15150 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:07:23.190596 15139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:23.190848 15139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:23.190881 15139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:23.192447 15139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:23.192516 15139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:23.192549 15139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:07:23.194298 15139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:35665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=39245
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:23.194590 15139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:23.194854 15139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:23.200081 15164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.200105 15165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.200098 15167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:23.200523 15139 server_base.cc:1034] running on GCE node
I20260619 12:07:23.200841 15139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:23.201104 15139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:23.202263 15139 hybrid_clock.cc:648] HybridClock initialized: now 1781870843202248 us; error 41 us; skew 500 ppm
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:23.205060 15139 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:07:23.205582 15139 webserver.cc:460] Webserver started at http://127.0.203.129:39245/ using document root <none> and password file <none>
I20260619 12:07:23.205814 15139 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:23.205894 15139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:23.207391 15139 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:23.208166 15174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.208465 15139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:23.208580 15139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "7ac1d4819ccf4ac4876fd2161995e41e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "de10647c5f7ee6c54bc49b304cbea5a1"
server_key_iv: "d95caa5b1af0e7f4b4a6214452d43c08"
server_key_version: "encryptionkey@0"
I20260619 12:07:23.208971 15139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:23.222715 15139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:23.224972 15139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:23.225116 15139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:23.225345 15139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:23.225803 15181 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:07:23.227232 15139 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:07:23.227279 15139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 12:07:23.227303 15139 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:07:23.227957 15139 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:07:23.228003 15139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 12:07:23.228067 15181 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap starting.
I20260619 12:07:23.236704 15139 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:35665
I20260619 12:07:23.237032 15288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:35665 every 8 connection(s)
I20260619 12:07:23.237322 15139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:07:23.240068 15181 log.cc:824] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:23.242110 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15139
I20260619 12:07:23.242439 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:33671
--local_ip_for_outbound_sockets=127.0.203.130
--tserver_master_addrs=127.0.203.190:38787
--webserver_port=37273
--webserver_interface=127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:23.242694 15181 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:07:23.243232 15181 tablet_bootstrap.cc:492] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap complete.
I20260619 12:07:23.244434 15181 ts_tablet_manager.cc:1397] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260619 12:07:23.245800 15181 raft_consensus.cc:348] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.246145 15181 raft_consensus.cc:729] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac1d4819ccf4ac4876fd2161995e41e, State: Initialized, Role: FOLLOWER
I20260619 12:07:23.246317 15181 consensus_queue.cc:260] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.246632 15181 ts_tablet_manager.cc:1428] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:23.252069 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.238170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:59851)
0619 12:07:23.238359 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.238415 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.239161 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.239811 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.239816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.240237 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:23.240541 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.240549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.242363 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.242365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.244010 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.244015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.244956 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.244958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.245107 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.245768 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:23.245778 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:23.247607 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:23.249924 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:23.249933 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:23.249936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:23.250262 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:23.250589 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:23.250590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:23.250590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:23.250741 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:23.251250 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:23.251256 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:23.251486 (+ 230us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.251623 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.251833 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260619 12:07:23.252151 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.238340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:59851 (local address 127.0.203.190:38787)
0619 12:07:23.238578 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.238591 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.238640 (+ 49us) server_negotiation.cc:407] Connection header received
0619 12:07:23.239497 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.239507 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.239580 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.239775 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:23.240686 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.242098 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.244133 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.244717 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.244799 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.247722 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:23.247727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:23.247727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:23.247739 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:23.249787 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.250370 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.250374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.250375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.250507 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.250825 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.250826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.250826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.251171 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:23.251254 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.251687 (+ 433us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.251850 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":97,"threads_started":1}
I20260619 12:07:23.253379 15289 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38787
I20260619 12:07:23.253515 15289 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:23.253746 15289 heartbeater.cc:511] Master 127.0.203.190:38787 requested a full tablet report, sending...
I20260619 12:07:23.254675 15101 ts_manager.cc:194] Registered new tserver with Master: 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665)
I20260619 12:07:23.256054 15101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:59851
I20260619 12:07:23.257165 15289 heartbeater.cc:503] Master 127.0.203.190:38787 was elected leader, sending a full tablet report...
I20260619 12:07:23.263608 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.255531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35665 (local address 127.0.0.1:45528)
0619 12:07:23.256048 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.256091 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.256262 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.259729 (+ 3467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.259733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.259737 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:23.259812 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.259814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.261307 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.261308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.262183 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.262186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.263275 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.263276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.263346 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.263411 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.263476 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.263529 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":111,"threads_started":1}
I20260619 12:07:23.263681 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.255861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45528 (local address 127.0.203.129:35665)
0619 12:07:23.259504 (+ 3643us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.259510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.259524 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:07:23.259567 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.259570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.259590 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.259693 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:23.259992 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.261205 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.262309 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.263157 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.263280 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.263348 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.263518 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.263573 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3596,"thread_start_us":77,"threads_started":1}
I20260619 12:07:23.264293 15223 tablet_service.cc:1514] Processing DeleteTablet for tablet f17440a91deb47b789b2412b2841ae06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:22 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:45528
I20260619 12:07:23.264909 15298 tablet_replica.cc:330] stopping tablet replica
I20260619 12:07:23.265029 15298 raft_consensus.cc:2229] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:07:23.265112 15298 raft_consensus.cc:2258] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:23.265489 15298 ts_tablet_manager.cc:1905] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:07:23.267709 15298 ts_tablet_manager.cc:1918] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:23.267781 15298 log.cc:1196] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/f17440a91deb47b789b2412b2841ae06
I20260619 12:07:23.268004 15298 ts_tablet_manager.cc:1939] T f17440a91deb47b789b2412b2841ae06 P 7ac1d4819ccf4ac4876fd2161995e41e: Deleting consensus metadata
I20260619 12:07:23.268584 15088 catalog_manager.cc:4841] TS 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665): tablet f17440a91deb47b789b2412b2841ae06 (table test-table [id=c0a589590e9a4de5ad3e97cccd06af9c]) successfully deleted
W20260619 12:07:23.334450 15294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:23.334681 15294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:23.334708 15294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:23.336128 15294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:23.336199 15294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:23.336259 15294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:07:23.338009 15294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:33671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=37273
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:23.338289 15294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:23.338548 15294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:23.344045 15304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.344063 15303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.344064 15306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:23.344103 15294 server_base.cc:1034] running on GCE node
I20260619 12:07:23.344525 15294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:23.344830 15294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:23.345999 15294 hybrid_clock.cc:648] HybridClock initialized: now 1781870843345988 us; error 31 us; skew 500 ppm
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:23.353474 15294 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:07:23.354117 15294 webserver.cc:460] Webserver started at http://127.0.203.130:37273/ using document root <none> and password file <none>
I20260619 12:07:23.354391 15294 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:23.354466 15294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:23.356295 15294 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:07:23.357198 15313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.357518 15294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:23.357630 15294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "47b1d7ead78a410291a57e2b926e201e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "ee09afb10915f126588fc9e7ac1c7b6e"
server_key_iv: "3af33e3840b21a309d6ef8760691597b"
server_key_version: "encryptionkey@0"
I20260619 12:07:23.358080 15294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:23.372067 15294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:23.374768 15294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:23.374912 15294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:23.375182 15294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:23.375581 15294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:23.375618 15294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.375651 15294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:23.375675 15294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.382730 15294 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:33671
I20260619 12:07:23.382776 15426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:33671 every 8 connection(s)
I20260619 12:07:23.383180 15294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:23.392287 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15294
I20260619 12:07:23.392675 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:46719
--local_ip_for_outbound_sockets=127.0.203.131
--tserver_master_addrs=127.0.203.190:38787
--webserver_port=44643
--webserver_interface=127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:23.393862 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.384045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:40969 (local address 127.0.203.190:38787)
0619 12:07:23.384102 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.384105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.384344 (+ 239us) server_negotiation.cc:407] Connection header received
0619 12:07:23.384995 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.384996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.385014 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.385070 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:23.386153 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.386715 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.387810 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.388053 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.388118 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.390784 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:23.390786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:23.390786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:23.390794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:23.392157 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.392768 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.392770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.392771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.392847 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.393145 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.393146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.393146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.393278 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:23.393336 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.393647 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.393756 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:07:23.394080 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.384015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:40969)
0619 12:07:23.384197 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.384242 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.384848 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.385148 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.385153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.385778 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:23.386051 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.386056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.386802 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.386804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.387732 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.387734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.388131 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.388132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.388267 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.388933 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:23.388944 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:23.390662 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:23.392267 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:23.392278 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:23.392281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:23.392680 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:23.392942 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:23.392944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:23.392944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:23.393082 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:23.393334 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:23.393342 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:23.393514 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.393619 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.393821 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260619 12:07:23.394649 15427 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38787
I20260619 12:07:23.394795 15427 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:23.395012 15427 heartbeater.cc:511] Master 127.0.203.190:38787 requested a full tablet report, sending...
I20260619 12:07:23.395496 15101 ts_manager.cc:194] Registered new tserver with Master: 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671)
I20260619 12:07:23.395926 15101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:40969
W20260619 12:07:23.481786 15430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:23.482056 15430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:23.482096 15430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:23.483407 15430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:23.483449 15430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:23.483489 15430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:07:23.485029 15430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:46719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=44643
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:23.485292 15430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:23.485548 15430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:07:23.491106 15430 server_base.cc:1034] running on GCE node
W20260619 12:07:23.491068 15436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.491089 15435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.491180 15438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:23.491596 15430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:23.491894 15430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:23.493088 15430 hybrid_clock.cc:648] HybridClock initialized: now 1781870843493071 us; error 43 us; skew 500 ppm
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:23.496091 15430 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:07:23.496637 15430 webserver.cc:460] Webserver started at http://127.0.203.131:44643/ using document root <none> and password file <none>
I20260619 12:07:23.496884 15430 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:23.496953 15430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:23.498589 15430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:23.499466 15445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.499692 15430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:23.499816 15430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "a1cd4b9cfbb144ce8180495c8e650bc0"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "0ffe0bedb01da35d3ce77420e29e8c92"
server_key_iv: "377dce400f62642dfc0fe03ab31dd8b5"
server_key_version: "encryptionkey@0"
I20260619 12:07:23.500223 15430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:23.521879 15430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:23.524057 15430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:23.524188 15430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:23.524415 15430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:23.524777 15430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:23.524823 15430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.524847 15430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:23.524874 15430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.531172 15430 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:46719
I20260619 12:07:23.531270 15558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:46719 every 8 connection(s)
I20260619 12:07:23.531616 15430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:23.538997 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15430
I20260619 12:07:23.542565 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.532493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:59005 (local address 127.0.203.190:38787)
0619 12:07:23.532572 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.532575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.532836 (+ 261us) server_negotiation.cc:407] Connection header received
0619 12:07:23.533385 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.533386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.533399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.533474 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:23.534539 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.535048 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.536189 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.536386 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.536443 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.539023 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:23.539025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:23.539025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:23.539034 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:23.540920 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.541516 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.541517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.541518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.541594 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.541855 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.541856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.541857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.542008 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:23.542080 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.542411 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.542487 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:07:23.542804 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.532411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:59005)
0619 12:07:23.532648 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.532694 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.533294 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.533544 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.533548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.534201 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:23.534452 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.534457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.535160 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.535160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.536099 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.536102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.536476 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.536477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.536602 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.537231 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:23.537241 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:23.538906 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:23.541026 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:23.541039 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:23.541042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:23.541433 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:23.541658 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:23.541659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:23.541659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:23.541789 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:23.542073 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:23.542076 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:23.542214 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.542336 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.542531 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260619 12:07:23.543428 15559 heartbeater.cc:347] Connected to a master server at 127.0.203.190:38787
I20260619 12:07:23.543531 15559 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:23.543726 15559 heartbeater.cc:511] Master 127.0.203.190:38787 requested a full tablet report, sending...
I20260619 12:07:23.544193 15101 ts_manager.cc:194] Registered new tserver with Master: a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719)
I20260619 12:07:23.544667 15101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:59005
I20260619 12:07:23.546636 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.539531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.0.203.190:38787)
0619 12:07:23.539662 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.539663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.539852 (+ 189us) server_negotiation.cc:407] Connection header received
0619 12:07:23.539932 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.539933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.539946 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.540000 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:23.540693 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.541339 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.542344 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.542629 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.542700 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.544237 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:23.544239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:23.544239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:23.544247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:23.545744 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.546057 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.546058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.546058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.546106 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:23.546243 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:23.546243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:23.546244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:23.546337 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:23.546457 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.546486 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.546556 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260619 12:07:23.547570 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:07:23.550530 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.547926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.0.203.190:38787)
0619 12:07:23.547997 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.547999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.548114 (+ 115us) server_negotiation.cc:407] Connection header received
0619 12:07:23.548166 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.548167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.548194 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.548266 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:07:23.548356 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.548835 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.549645 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.549862 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.549923 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.550106 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:07:23.550206 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:07:23.550273 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.550432 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.550454 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:07:23.550882 15101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:07:23.552829 15101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:07:23.555485 15223 tablet_service.cc:1467] Processing CreateTablet for tablet 7cf1bc7adadd4686a682132b4c2c1ae1 (DEFAULT_TABLE table=test-table [id=15e9dd6084eb45dcb2602d3158719dc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:23.555773 15223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cf1bc7adadd4686a682132b4c2c1ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:23.557643 15181 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap starting.
I20260619 12:07:23.558024 15181 tablet_bootstrap.cc:654] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:23.558575 15181 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: No bootstrap required, opened a new log
I20260619 12:07:23.558630 15181 ts_tablet_manager.cc:1397] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:23.558768 15181 raft_consensus.cc:348] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.558849 15181 raft_consensus.cc:374] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:23.558903 15181 raft_consensus.cc:729] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac1d4819ccf4ac4876fd2161995e41e, State: Initialized, Role: FOLLOWER
I20260619 12:07:23.558966 15181 consensus_queue.cc:260] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.559067 15181 ts_tablet_manager.cc:1428] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 12:07:23.559230 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.554991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:33671 (local address 127.0.0.1:58844)
0619 12:07:23.555230 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.555243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.555310 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.555674 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.555675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.555678 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:23.555749 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.555751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.556712 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.556714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.557712 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.557715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.558703 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.558704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.558770 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.558775 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.558993 (+ 218us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.559056 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":29,"thread_start_us":135,"threads_started":1}
I20260619 12:07:23.559597 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.555193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.0.203.130:33671)
0619 12:07:23.555465 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.555471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.555481 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:07:23.555525 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.555528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.555544 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.555660 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:23.555841 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.556610 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.557825 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.558625 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.558686 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.558766 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.559418 (+ 652us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.559476 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":125,"threads_started":1}
I20260619 12:07:23.560346 15361 tablet_service.cc:1467] Processing CreateTablet for tablet 7cf1bc7adadd4686a682132b4c2c1ae1 (DEFAULT_TABLE table=test-table [id=15e9dd6084eb45dcb2602d3158719dc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:23.560554 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.554983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:46719 (local address 127.0.0.1:60604)
0619 12:07:23.555033 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.555046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.555088 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.555859 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.555860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.555861 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:23.555896 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.555897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.557194 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.557195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.558211 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.558214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.560256 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.560258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.560331 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.560335 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.560384 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.560465 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 12:07:23.560567 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.555279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.0.203.131:46719)
0619 12:07:23.555525 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.555529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.555540 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:07:23.555577 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.555579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.555593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.555676 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:23.555988 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.557071 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.558304 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.560107 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.560182 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.560250 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.560407 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.560464 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":151,"threads_started":1}
I20260619 12:07:23.560730 15361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cf1bc7adadd4686a682132b4c2c1ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:23.561275 15493 tablet_service.cc:1467] Processing CreateTablet for tablet 7cf1bc7adadd4686a682132b4c2c1ae1 (DEFAULT_TABLE table=test-table [id=15e9dd6084eb45dcb2602d3158719dc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:23.561601 15493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cf1bc7adadd4686a682132b4c2c1ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:23.564045 15569 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Bootstrap starting.
I20260619 12:07:23.564872 15569 tablet_bootstrap.cc:654] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:23.564901 15570 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Bootstrap starting.
I20260619 12:07:23.565441 15569 log.cc:824] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:23.565660 15570 tablet_bootstrap.cc:654] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:23.566215 15569 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: No bootstrap required, opened a new log
I20260619 12:07:23.566217 15570 log.cc:824] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:23.566301 15569 ts_tablet_manager.cc:1397] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:07:23.566993 15570 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: No bootstrap required, opened a new log
I20260619 12:07:23.567057 15570 ts_tablet_manager.cc:1397] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:07:23.568104 15569 raft_consensus.cc:348] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.568269 15569 raft_consensus.cc:374] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:23.568305 15569 raft_consensus.cc:729] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b1d7ead78a410291a57e2b926e201e, State: Initialized, Role: FOLLOWER
I20260619 12:07:23.568454 15569 consensus_queue.cc:260] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.568643 15570 raft_consensus.cc:348] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.568776 15569 ts_tablet_manager.cc:1428] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:07:23.568815 15570 raft_consensus.cc:374] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:23.568825 15427 heartbeater.cc:503] Master 127.0.203.190:38787 was elected leader, sending a full tablet report...
I20260619 12:07:23.568853 15570 raft_consensus.cc:729] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cd4b9cfbb144ce8180495c8e650bc0, State: Initialized, Role: FOLLOWER
I20260619 12:07:23.568969 15570 consensus_queue.cc:260] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.569278 15570 ts_tablet_manager.cc:1428] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:07:23.569416 15559 heartbeater.cc:503] Master 127.0.203.190:38787 was elected leader, sending a full tablet report...
I20260619 12:07:23.618597 15574 raft_consensus.cc:482] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:23.618743 15574 raft_consensus.cc:504] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.619099 15574 leader_election.cc:288] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665)
I20260619 12:07:23.622466 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.619441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:49199 (local address 127.0.203.129:35665)
0619 12:07:23.619486 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.619488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.619707 (+ 219us) server_negotiation.cc:407] Connection header received
0619 12:07:23.619741 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.619742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.619751 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.619796 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:23.620006 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.620667 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.621705 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.622096 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.622170 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.622180 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.622341 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.622385 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:07:23.622469 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.619609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:37843 (local address 127.0.203.130:33671)
0619 12:07:23.619674 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.619676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.619688 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:07:23.619720 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.619721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.619729 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.619789 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:23.620018 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.620667 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.621709 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.622112 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.622181 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.622192 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.622352 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.622396 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:07:23.622517 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.619333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:33671 (local address 127.0.203.131:37843)
0619 12:07:23.619414 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.619430 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.619516 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.619808 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.619810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.619813 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:23.619933 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.619936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.620748 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.620749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.621600 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.621602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.622198 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.622200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.622276 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.622280 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.622338 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.622393 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":27}
I20260619 12:07:23.622545 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.619333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35665 (local address 127.0.203.131:49199)
0619 12:07:23.619585 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.619594 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.619675 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:23.619811 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:23.619812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:23.619814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:23.619937 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.619940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.620769 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.620770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.621602 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:23.621604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.622184 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.622185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:23.622262 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.622266 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:07:23.622313 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:07:23.622365 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":49,"thread_start_us":132,"threads_started":1}
I20260619 12:07:23.622983 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf1bc7adadd4686a682132b4c2c1ae1" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b1d7ead78a410291a57e2b926e201e" is_pre_election: true
I20260619 12:07:23.623180 15381 raft_consensus.cc:2454] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 0.
I20260619 12:07:23.623459 15448 leader_election.cc:302] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0; no voters:
I20260619 12:07:23.623394 15243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf1bc7adadd4686a682132b4c2c1ae1" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" is_pre_election: true
I20260619 12:07:23.623600 15243 raft_consensus.cc:2454] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 0.
I20260619 12:07:23.623764 15574 raft_consensus.cc:2781] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:07:23.623840 15574 raft_consensus.cc:482] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:07:23.623857 15574 raft_consensus.cc:3037] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:23.624651 15574 raft_consensus.cc:504] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.624809 15574 leader_election.cc:288] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 election: Requested vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665)
I20260619 12:07:23.625015 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf1bc7adadd4686a682132b4c2c1ae1" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b1d7ead78a410291a57e2b926e201e"
I20260619 12:07:23.625041 15243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf1bc7adadd4686a682132b4c2c1ae1" candidate_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac1d4819ccf4ac4876fd2161995e41e"
I20260619 12:07:23.625131 15243 raft_consensus.cc:3037] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:23.625134 15381 raft_consensus.cc:3037] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:23.626084 15243 raft_consensus.cc:2454] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 1.
I20260619 12:07:23.626084 15381 raft_consensus.cc:2454] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cd4b9cfbb144ce8180495c8e650bc0 in term 1.
I20260619 12:07:23.626298 15448 leader_election.cc:302] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0; no voters:
I20260619 12:07:23.626453 15574 raft_consensus.cc:2781] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:23.626668 15574 raft_consensus.cc:686] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 LEADER]: Becoming Leader. State: Replica: a1cd4b9cfbb144ce8180495c8e650bc0, State: Running, Role: LEADER
I20260619 12:07:23.626785 15574 consensus_queue.cc:237] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.627924 15101 catalog_manager.cc:5495] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } health_report { overall_health: UNKNOWN } } }
W20260619 12:07:23.633834 15428 tablet.cc:2369] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:07:23.640219 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:23.637715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.0.203.131:46719)
0619 12:07:23.637799 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:23.637802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:23.637811 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:07:23.637849 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:23.637850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:23.637869 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:23.637942 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:07:23.638073 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.638552 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.639328 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:23.639609 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:23.639681 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:23.639847 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:07:23.639962 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:07:23.640044 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:23.640086 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 12:07:23.640107 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:07:23.642546 15243 raft_consensus.cc:1261] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Refusing update from remote peer a1cd4b9cfbb144ce8180495c8e650bc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:23.642518 15381 raft_consensus.cc:1261] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Refusing update from remote peer a1cd4b9cfbb144ce8180495c8e650bc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:23.642982 15576 consensus_queue.cc:1035] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:23.643199 15574 consensus_queue.cc:1035] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:23.646706 15579 mvcc.cc:205] Tried to move back new op lower bound from 7298542975556173824 to 7298542975496458240. Current Snapshot: MvccSnapshot[applied={T|T < 7298542975556173824}]
I20260619 12:07:23.648705 15101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:23.648814 15101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:23.649626 15101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Sending DeleteTablet for 3 replicas of tablet 7cf1bc7adadd4686a682132b4c2c1ae1
I20260619 12:07:23.649983 15361 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cf1bc7adadd4686a682132b4c2c1ae1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:23 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:58844
I20260619 12:07:23.650020 15223 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cf1bc7adadd4686a682132b4c2c1ae1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:23 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:45528
I20260619 12:07:23.649998 15493 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cf1bc7adadd4686a682132b4c2c1ae1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:23 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:60604
I20260619 12:07:23.650113 15298 tablet_replica.cc:330] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: stopping tablet replica
I20260619 12:07:23.650171 15298 raft_consensus.cc:2229] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:07:23.650234 15588 tablet_replica.cc:330] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: stopping tablet replica
I20260619 12:07:23.650241 15298 raft_consensus.cc:2258] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:23.650327 15588 raft_consensus.cc:2229] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:07:23.650446 15588 raft_consensus.cc:2258] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:23.650467 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 15139
I20260619 12:07:23.651648 15588 ts_tablet_manager.cc:1905] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:07:23.653412 15589 tablet_replica.cc:330] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: stopping tablet replica
I20260619 12:07:23.653545 15589 raft_consensus.cc:2229] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:07:23.653710 15589 raft_consensus.cc:2258] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:23.653790 15588 ts_tablet_manager.cc:1918] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:23.653940 15588 log.cc:1196] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/7cf1bc7adadd4686a682132b4c2c1ae1
I20260619 12:07:23.654297 15588 ts_tablet_manager.cc:1939] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 47b1d7ead78a410291a57e2b926e201e: Deleting consensus metadata
I20260619 12:07:23.654922 15087 catalog_manager.cc:4841] TS 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671): tablet 7cf1bc7adadd4686a682132b4c2c1ae1 (table test-table [id=15e9dd6084eb45dcb2602d3158719dc5]) successfully deleted
I20260619 12:07:23.655833 15589 ts_tablet_manager.cc:1905] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:07:23.657305 15088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:35665 (error 108)
I20260619 12:07:23.657943 15589 ts_tablet_manager.cc:1918] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:23.658126 15589 log.cc:1196] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/7cf1bc7adadd4686a682132b4c2c1ae1
W20260619 12:07:23.658124 15296 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:07:23.657923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:35665 (local address 127.0.0.1:45536)
0619 12:07:23.658000 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.658063 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:35665: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 12:07:23.658361 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 15294
W20260619 12:07:23.658260 15088 catalog_manager.cc:4584] TS 7ac1d4819ccf4ac4876fd2161995e41e (127.0.203.129:35665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cf1bc7adadd4686a682132b4c2c1ae1: Network error: Client connection negotiation failed: client connection to 127.0.203.129:35665: connect: Connection reset by peer (error 104)
I20260619 12:07:23.662786 15589 ts_tablet_manager.cc:1939] T 7cf1bc7adadd4686a682132b4c2c1ae1 P a1cd4b9cfbb144ce8180495c8e650bc0: Deleting consensus metadata
I20260619 12:07:23.663478 15085 catalog_manager.cc:4841] TS a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719): tablet 7cf1bc7adadd4686a682132b4c2c1ae1 (table test-table [id=15e9dd6084eb45dcb2602d3158719dc5]) successfully deleted
I20260619 12:07:23.663707 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 15430
I20260619 12:07:23.669253 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 15068
I20260619 12:07:23.675125 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38787
--webserver_interface=127.0.203.190
--webserver_port=33667
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:38787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:23.764701 15591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:23.764971 15591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:23.764995 15591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:23.766268 15591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:23.766314 15591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:23.766326 15591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:23.766338 15591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:23.766350 15591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:23.767982 15591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:38787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:38787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=33667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:23.768249 15591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:23.768513 15591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:23.774024 15597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.774024 15596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:23.774276 15591 server_base.cc:1034] running on GCE node
W20260619 12:07:23.774075 15599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:23.774739 15591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:23.775097 15591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:23.776273 15591 hybrid_clock.cc:648] HybridClock initialized: now 1781870843776216 us; error 77 us; skew 500 ppm
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:23.779281 15591 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:23.779879 15591 webserver.cc:460] Webserver started at http://127.0.203.190:33667/ using document root <none> and password file <none>
I20260619 12:07:23.780131 15591 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:23.780184 15591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:23.781778 15591 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:23.782562 15606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.782821 15591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:23.782922 15591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "47e81fc6b76f4319b71b47fba0121014"
format_stamp: "Formatted at 2026-06-19 12:07:21 on dist-test-slave-44vj"
server_key: "3caabd9067aedc837fe3b150b30f45bf"
server_key_iv: "c9ae127aa0a1af43b26f574bd53d1b9d"
server_key_version: "encryptionkey@0"
I20260619 12:07:23.783429 15591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:23.795953 15591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:23.798475 15591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:23.798633 15591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:23.803974 15591 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:38787
I20260619 12:07:23.804044 15658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:38787 every 8 connection(s)
I20260619 12:07:23.804510 15591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:23.806573 15659 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:07:23.807353 15659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Bootstrap starting.
I20260619 12:07:23.811149 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15591
I20260619 12:07:23.811559 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:35665
--local_ip_for_outbound_sockets=127.0.203.129
--tserver_master_addrs=127.0.203.190:38787
--webserver_port=39245
--webserver_interface=127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:23.817065 15659 log.cc:824] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:23.820700 15659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:07:23.821106 15659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014: Bootstrap complete.
I20260619 12:07:23.824863 15659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.825218 15659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e81fc6b76f4319b71b47fba0121014, State: Initialized, Role: FOLLOWER
I20260619 12:07:23.825403 15659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.825546 15659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:07:23.825606 15659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:07:23.825649 15659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:07:23.827139 15659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.827286 15659 leader_election.cc:302] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47e81fc6b76f4319b71b47fba0121014; no voters:
I20260619 12:07:23.827517 15659 leader_election.cc:288] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 12:07:23.827643 15663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:07:23.827833 15659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:23.827943 15663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [term 3 LEADER]: Becoming Leader. State: Replica: 47e81fc6b76f4319b71b47fba0121014, State: Running, Role: LEADER
I20260619 12:07:23.829409 15663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } }
I20260619 12:07:23.830233 15664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "47e81fc6b76f4319b71b47fba0121014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } } }
I20260619 12:07:23.830361 15664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:23.830271 15665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47e81fc6b76f4319b71b47fba0121014. Latest consensus state: current_term: 3 leader_uuid: "47e81fc6b76f4319b71b47fba0121014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e81fc6b76f4319b71b47fba0121014" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 38787 } } }
I20260619 12:07:23.830432 15665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e81fc6b76f4319b71b47fba0121014 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:23.830679 15678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:23.831748 15678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:07:23.831861 15678 catalog_manager.cc:1259] Loaded cluster ID: 31f65254bd2b48a88a08127dac1d8212
I20260619 12:07:23.831894 15678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 12:07:23.832170 15678 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f80880799ad google::LogMessage::Fail() at ??:0
@ 0x7f808807c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f80880794de google::LogMessage::Flush() at ??:0
@ 0x7f808807aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f808ff9e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f808ffa78b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8089f470d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8089f3c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f808ada06db start_thread at ??:0
@ 0x7f8086b7761f clone at ??:0
W20260619 12:07:23.919209 15661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:23.919507 15661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:23.919541 15661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:23.920848 15661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:23.920907 15661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:23.920935 15661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:07:23.922549 15661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:35665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=39245
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:23.922811 15661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:23.923060 15661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:23.928252 15698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.928418 15695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:23.928261 15696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:23.928660 15661 server_base.cc:1034] running on GCE node
I20260619 12:07:23.928912 15661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:23.929176 15661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:23.930382 15661 hybrid_clock.cc:648] HybridClock initialized: now 1781870843930370 us; error 29 us; skew 500 ppm
Jun 19 12:07:23 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:23.932973 15661 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:07:23.933501 15661 webserver.cc:460] Webserver started at http://127.0.203.129:39245/ using document root <none> and password file <none>
I20260619 12:07:23.933735 15661 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:23.933789 15661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:23.935413 15661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:23.936269 15705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:23.936620 15661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:23.936775 15661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "7ac1d4819ccf4ac4876fd2161995e41e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "de10647c5f7ee6c54bc49b304cbea5a1"
server_key_iv: "d95caa5b1af0e7f4b4a6214452d43c08"
server_key_version: "encryptionkey@0"
I20260619 12:07:23.937409 15661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:23.954166 15661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:23.957161 15661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:23.957300 15661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:23.957602 15661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:23.958158 15712 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:07:23.960434 15661 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:07:23.960507 15661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260619 12:07:23.960547 15661 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:07:23.961385 15661 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:07:23.961421 15661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 12:07:23.961562 15712 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap starting.
I20260619 12:07:23.969381 15661 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:35665
I20260619 12:07:23.969823 15661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:07:23.971686 15819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:35665 every 8 connection(s)
I20260619 12:07:23.973269 15712 log.cc:824] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:23.976933 15712 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:07:23.977309 15712 tablet_bootstrap.cc:492] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Bootstrap complete.
I20260619 12:07:23.978394 15712 ts_tablet_manager.cc:1397] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.011s
I20260619 12:07:23.979688 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15661
I20260619 12:07:23.979986 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:33671
--local_ip_for_outbound_sockets=127.0.203.130
--tserver_master_addrs=127.0.203.190:38787
--webserver_port=37273
--webserver_interface=127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:23.980137 15712 raft_consensus.cc:348] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.980537 15712 raft_consensus.cc:729] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac1d4819ccf4ac4876fd2161995e41e, State: Initialized, Role: FOLLOWER
I20260619 12:07:23.980787 15712 consensus_queue.cc:260] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:23.981164 15712 ts_tablet_manager.cc:1428] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
W20260619 12:07:24.105520 15825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:24.105759 15825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:24.105786 15825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:24.107066 15825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:24.107120 15825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:24.107148 15825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:07:24.108767 15825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:33671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=37273
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:24.109014 15825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:24.109256 15825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:24.116395 15833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:24.116493 15825 server_base.cc:1034] running on GCE node
W20260619 12:07:24.116603 15830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:24.116636 15831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:24.117093 15825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:24.117441 15825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:24.118633 15825 hybrid_clock.cc:648] HybridClock initialized: now 1781870844118612 us; error 38 us; skew 500 ppm
W20260619 12:07:24.120625 15822 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:07:23.970621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:35805)
0619 12:07:23.970808 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:23.970857 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:23.971450 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:24.120352 (+148902us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
W20260619 12:07:24.120923 15820 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:38787 (0 consecutive failures): Network error: Failed to ping master at 127.0.203.190:38787: Client connection negotiation failed: client connection to 127.0.203.190:38787: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Jun 19 12:07:24 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:24.121310 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.121156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:56179)
0619 12:07:24.121229 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.121272 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:07:24.121634 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.121555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:32989)
0619 12:07:24.121591 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.121613 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 12:07:24.121625 15825 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
W20260619 12:07:24.121748 15820 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:07:24.122180 15825 webserver.cc:460] Webserver started at http://127.0.203.130:37273/ using document root <none> and password file <none>
I20260619 12:07:24.122440 15825 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:24.122495 15825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:24.124085 15825 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:07:24.124917 15840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:24.125140 15825 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:24.125231 15825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "47b1d7ead78a410291a57e2b926e201e"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "ee09afb10915f126588fc9e7ac1c7b6e"
server_key_iv: "3af33e3840b21a309d6ef8760691597b"
server_key_version: "encryptionkey@0"
I20260619 12:07:24.125648 15825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:24.134716 15825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:24.137290 15825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:24.137461 15825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:24.137676 15825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:24.137998 15825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:24.138037 15825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:24.138073 15825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:24.138101 15825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:24.144908 15825 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:33671
I20260619 12:07:24.145035 15953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:33671 every 8 connection(s)
I20260619 12:07:24.145401 15825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:07:24.147025 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.146324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:32819)
0619 12:07:24.146586 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.146837 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":110,"threads_started":1}
W20260619 12:07:24.147643 15954 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:38787 (0 consecutive failures): Network error: Failed to ping master at 127.0.203.190:38787: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
I20260619 12:07:24.147805 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15825
I20260619 12:07:24.148031 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.147913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:52901)
0619 12:07:24.147959 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.147998 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 12:07:24.148156 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:46719
--local_ip_for_outbound_sockets=127.0.203.131
--tserver_master_addrs=127.0.203.190:38787
--webserver_port=44643
--webserver_interface=127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:07:24.148320 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.148234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:55095)
0619 12:07:24.148272 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.148296 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 12:07:24.148488 15954 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:07:24.237879 15957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:24.238143 15957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:24.238170 15957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:24.239511 15957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:24.239570 15957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:24.239600 15957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:07:24.241309 15957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:46719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=44643
--tserver_master_addrs=127.0.203.190:38787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:24.241619 15957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:24.241923 15957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:24.247692 15965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:24.247774 15957 server_base.cc:1034] running on GCE node
W20260619 12:07:24.247812 15963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:24.247812 15962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:24.248270 15957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:24.248553 15957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:24.249780 15957 hybrid_clock.cc:648] HybridClock initialized: now 1781870844249752 us; error 37 us; skew 500 ppm
Jun 19 12:07:24 dist-test-slave-44vj krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:24.253156 15957 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:07:24.253840 15957 webserver.cc:460] Webserver started at http://127.0.203.131:44643/ using document root <none> and password file <none>
I20260619 12:07:24.254143 15957 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:24.254195 15957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:24.255880 15957 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:24.256949 15972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:24.257167 15957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:24.257265 15957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "a1cd4b9cfbb144ce8180495c8e650bc0"
format_stamp: "Formatted at 2026-06-19 12:07:22 on dist-test-slave-44vj"
server_key: "0ffe0bedb01da35d3ce77420e29e8c92"
server_key_iv: "377dce400f62642dfc0fe03ab31dd8b5"
server_key_version: "encryptionkey@0"
I20260619 12:07:24.257787 15957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:24.277899 15957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:24.280561 15957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:24.280704 15957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:24.280932 15957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:24.281315 15957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:24.281358 15957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:24.281386 15957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:24.281399 15957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:24.288645 15957 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:46719
I20260619 12:07:24.288739 16085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:46719 every 8 connection(s)
I20260619 12:07:24.289111 15957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:07:24.290409 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.289905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:35259)
0619 12:07:24.290130 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.290234 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
W20260619 12:07:24.290869 16086 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:38787 (0 consecutive failures): Network error: Failed to ping master at 127.0.203.190:38787: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
I20260619 12:07:24.291195 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.291089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:48735)
0619 12:07:24.291143 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.291177 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:07:24.291664 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:24.291501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:46693)
0619 12:07:24.291593 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:24.291639 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260619 12:07:24.291822 16086 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:07:24.294816 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 15957
I20260619 12:07:25.122758 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:25.122224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:47207)
0619 12:07:25.122474 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:25.122606 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":75,"threads_started":1}
I20260619 12:07:25.149300 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:25.148887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:49021)
0619 12:07:25.149134 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:25.149198 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
I20260619 12:07:25.160629 16092 raft_consensus.cc:482] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:25.160797 16092 raft_consensus.cc:504] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:25.161199 16092 leader_election.cc:288] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719)
Jun 19 12:07:25 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.131@KRBTEST.COM
Jun 19 12:07:25 dist-test-slave-44vj krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.130@KRBTEST.COM
I20260619 12:07:25.173210 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:25.161538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:48255 (local address 127.0.203.130:33671)
0619 12:07:25.161765 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:25.161778 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:25.161816 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:07:25.162340 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:25.162342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:25.162391 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:25.162539 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:25.163562 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.164589 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.165812 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.166264 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.166376 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:25.169534 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:25.169540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:25.169540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:25.169553 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:25.171443 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:25.171918 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:25.171921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:25.171921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:25.172050 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:25.172257 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:25.172258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:25.172259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:25.172553 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:25.172642 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:25.172928 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 12:07:25.173076 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":95,"threads_started":1}
I20260619 12:07:25.173202 16095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:25.161584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:38957 (local address 127.0.203.131:46719)
0619 12:07:25.161733 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:25.161749 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:25.161782 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:07:25.162332 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:25.162346 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:25.162412 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:25.162564 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:25.163578 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.164601 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.165805 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.166256 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.166376 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:25.169232 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:25.169238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:25.169239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:25.169255 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:25.171226 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:25.171802 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:25.171806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:25.171807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:25.171935 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:25.172227 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:25.172228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:25.172228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:25.172530 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:25.172598 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:25.172937 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 12:07:25.173079 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":51,"threads_started":1}
I20260619 12:07:25.173199 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:25.161383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:46719 (local address 127.0.203.129:38957)
0619 12:07:25.161448 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:25.161461 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:25.161727 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:25.162633 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:25.162636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:25.163135 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:25.163411 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:25.163421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.164721 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.164722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:25.165680 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:25.165683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.166371 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.166373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:25.166502 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:25.167207 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:25.167212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:25.169027 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:25.171323 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:25.171334 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:25.171338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:25.171713 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:25.172012 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:25.172012 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:25.172013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:25.172145 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:25.172608 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:25.172615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:25.172755 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 12:07:25.172844 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 12:07:25.173089 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":24}
I20260619 12:07:25.173319 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:25.161379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:33671 (local address 127.0.203.129:48255)
0619 12:07:25.161636 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:25.161650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:25.161756 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:25.162633 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:25.162637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:25.163134 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:25.163413 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:25.163423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.164733 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.164734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:25.165682 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:25.165687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:25.166376 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:25.166377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:25.166506 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:25.167170 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:25.167180 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:25.169414 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:25.171539 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:25.171541 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:25.171542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:25.171801 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:25.172123 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:25.172124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:25.172124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:25.172185 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:25.172635 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:25.172637 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:25.172754 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 12:07:25.172855 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:07:25.173177 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":237,"thread_start_us":72,"threads_started":1}
W20260619 12:07:25.174297 15706 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
W20260619 12:07:25.174777 15708 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
I20260619 12:07:25.174834 15708 leader_election.cc:302] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ac1d4819ccf4ac4876fd2161995e41e; no voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0
I20260619 12:07:25.174969 16092 raft_consensus.cc:2726] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:07:25.292659 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:25.292274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:43779)
0619 12:07:25.292514 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:25.292586 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":118,"threads_started":1}
I20260619 12:07:26.123915 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:26.123600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:41449)
0619 12:07:26.123789 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:26.123854 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260619 12:07:26.150162 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:26.149899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:44065)
0619 12:07:26.150046 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:26.150109 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260619 12:07:26.293574 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:26.293249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:60891)
0619 12:07:26.293435 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:26.293490 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260619 12:07:27.050031 16100 raft_consensus.cc:482] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:27.050141 16100 raft_consensus.cc:504] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:27.050364 16100 leader_election.cc:288] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719)
W20260619 12:07:27.050822 15708 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
W20260619 12:07:27.050917 15706 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
I20260619 12:07:27.051004 15706 leader_election.cc:302] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ac1d4819ccf4ac4876fd2161995e41e; no voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0
I20260619 12:07:27.051106 16100 raft_consensus.cc:2726] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:07:27.124821 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:27.124428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:46167)
0619 12:07:27.124698 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:27.124764 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":158,"threads_started":1}
I20260619 12:07:27.151119 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:27.150811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:43427)
0619 12:07:27.150982 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:27.151055 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260619 12:07:27.294570 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:27.294204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:50531)
0619 12:07:27.294381 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:27.294455 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260619 12:07:28.125725 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:28.125403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:36617)
0619 12:07:28.125609 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:28.125677 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
I20260619 12:07:28.152074 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:28.151741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:52255)
0619 12:07:28.151943 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:28.151985 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260619 12:07:28.295466 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:28.295080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:47459)
0619 12:07:28.295312 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:28.295404 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":98,"threads_started":1}
I20260619 12:07:29.126701 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:29.126330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:36121)
0619 12:07:29.126588 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:29.126653 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":142,"threads_started":1}
I20260619 12:07:29.152994 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:29.152642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:36697)
0619 12:07:29.152861 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:29.152922 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260619 12:07:29.296340 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:29.296041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:34151)
0619 12:07:29.296224 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:29.296277 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260619 12:07:29.454466 16110 raft_consensus.cc:482] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:29.454577 16110 raft_consensus.cc:504] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:29.454756 16110 leader_election.cc:288] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719)
W20260619 12:07:29.455276 15708 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
W20260619 12:07:29.455386 15706 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
I20260619 12:07:29.455452 15706 leader_election.cc:302] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ac1d4819ccf4ac4876fd2161995e41e; no voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0
I20260619 12:07:29.455582 16110 raft_consensus.cc:2726] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:07:30.127849 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:30.127512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:58001)
0619 12:07:30.127698 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:30.127775 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260619 12:07:30.153784 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:30.153489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:40425)
0619 12:07:30.153649 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:30.153705 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260619 12:07:30.297200 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:30.296919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:37445)
0619 12:07:30.297094 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:30.297150 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260619 12:07:31.128851 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:31.128570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:56767)
0619 12:07:31.128741 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:31.128789 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
I20260619 12:07:31.154582 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:31.154249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:58277)
0619 12:07:31.154457 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:31.154528 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":83,"threads_started":1}
I20260619 12:07:31.298146 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:31.297828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:53375)
0619 12:07:31.298015 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:31.298059 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":116,"threads_started":1}
I20260619 12:07:32.129920 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:32.129539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:57363)
0619 12:07:32.129792 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:32.129847 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":164,"threads_started":1}
I20260619 12:07:32.155543 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:32.155184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:41477)
0619 12:07:32.155427 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:32.155495 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":108,"threads_started":1}
I20260619 12:07:32.299028 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:32.298707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:35795)
0619 12:07:32.298909 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:32.298973 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
I20260619 12:07:33.130919 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:33.130612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:43527)
0619 12:07:33.130804 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:33.130856 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":106,"threads_started":1}
I20260619 12:07:33.156307 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:33.156066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:44363)
0619 12:07:33.156219 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:33.156258 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260619 12:07:33.186849 16122 raft_consensus.cc:482] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:33.186985 16122 raft_consensus.cc:504] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:33.187192 16122 leader_election.cc:288] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719)
W20260619 12:07:33.187796 15706 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
W20260619 12:07:33.187892 15708 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
I20260619 12:07:33.187960 15708 leader_election.cc:302] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ac1d4819ccf4ac4876fd2161995e41e; no voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0
I20260619 12:07:33.188056 16122 raft_consensus.cc:2726] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:07:33.299954 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:33.299695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:52743)
0619 12:07:33.299857 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:33.299888 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260619 12:07:34.131918 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:34.131574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:36747)
0619 12:07:34.131788 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:34.131855 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260619 12:07:34.157305 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:34.156897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:40337)
0619 12:07:34.157186 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:34.157248 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":186,"threads_started":1}
I20260619 12:07:34.300827 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:34.300481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:56809)
0619 12:07:34.300696 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:34.300769 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":101,"threads_started":1}
I20260619 12:07:35.132987 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:35.132628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:46179)
0619 12:07:35.132849 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:35.132930 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260619 12:07:35.158288 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:35.158010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:37931)
0619 12:07:35.158191 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:35.158227 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260619 12:07:35.301894 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:35.301513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:47255)
0619 12:07:35.301761 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:35.301824 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":158,"threads_started":1}
I20260619 12:07:36.133898 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:36.133567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:60203)
0619 12:07:36.133761 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:36.133833 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":68,"threads_started":1}
I20260619 12:07:36.159240 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:36.158951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:54491)
0619 12:07:36.159143 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:36.159183 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260619 12:07:36.302947 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:36.302631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:38715)
0619 12:07:36.302826 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:36.302896 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
I20260619 12:07:36.607415 16133 raft_consensus.cc:482] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:36.607575 16133 raft_consensus.cc:504] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47b1d7ead78a410291a57e2b926e201e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 33671 } } peers { permanent_uuid: "a1cd4b9cfbb144ce8180495c8e650bc0" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46719 } } peers { permanent_uuid: "7ac1d4819ccf4ac4876fd2161995e41e" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 35665 } }
I20260619 12:07:36.607785 16133 leader_election.cc:288] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671), a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719)
W20260619 12:07:36.608317 15708 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47b1d7ead78a410291a57e2b926e201e (127.0.203.130:33671): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
W20260619 12:07:36.608448 15706 leader_election.cc:341] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1cd4b9cfbb144ce8180495c8e650bc0 (127.0.203.131:46719): Not found: Tablet not found: 7cf1bc7adadd4686a682132b4c2c1ae1
I20260619 12:07:36.608520 15706 leader_election.cc:302] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ac1d4819ccf4ac4876fd2161995e41e; no voters: 47b1d7ead78a410291a57e2b926e201e, a1cd4b9cfbb144ce8180495c8e650bc0
I20260619 12:07:36.608639 16133 raft_consensus.cc:2726] T 7cf1bc7adadd4686a682132b4c2c1ae1 P 7ac1d4819ccf4ac4876fd2161995e41e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:07:37.134866 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:37.134531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:52123)
0619 12:07:37.134748 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:37.134812 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
I20260619 12:07:37.160137 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:37.159878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:43973)
0619 12:07:37.160033 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:37.160075 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260619 12:07:37.303889 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:37.303560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:43615)
0619 12:07:37.303776 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:37.303834 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":106,"threads_started":1}
I20260619 12:07:38.135763 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:38.135485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:47239)
0619 12:07:38.135657 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:38.135717 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260619 12:07:38.161010 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:38.160685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:56341)
0619 12:07:38.160864 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:38.160942 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260619 12:07:38.304862 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:38.304594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:60761)
0619 12:07:38.304760 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:38.304795 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260619 12:07:39.136793 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:39.136447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.129:37983)
0619 12:07:39.136638 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:39.136728 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":78,"threads_started":1}
I20260619 12:07:39.162139 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:39.161769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.130:53157)
0619 12:07:39.162000 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:39.162081 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
I20260619 12:07:39.302001 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 15661
I20260619 12:07:39.305933 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:39.305657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:38787 (local address 127.0.203.131:34359)
0619 12:07:39.305825 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:39.305866 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:38787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260619 12:07:39.307194 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 15825
I20260619 12:07:39.311399 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 15957
2026-06-19T12:07:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18525 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:07:39 dist-test-slave-44vj krb5kdc[16145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:07:39 dist-test-slave-44vj krb5kdc[16145](info): set up 2 sockets
Jun 19 12:07:39 dist-test-slave-44vj krb5kdc[16145](info): commencing operation
krb5kdc: starting...
W20260619 12:07:40.279731 814 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:07:40 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:07:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:07:40.408169 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:45807
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:40.498741 16158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:40.498971 16158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:40.499003 16158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:40.500264 16158 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:07:40.500324 16158 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:07:40.500341 16158 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:07:40.500357 16158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:07:40.500375 16158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:40.500391 16158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:40.500412 16158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:07:40.500435 16158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:07:40.502029 16158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:45807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:40.502300 16158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:40.502529 16158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:40.507666 16166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:40.507725 16164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:40.507666 16163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:40.507715 16158 server_base.cc:1034] running on GCE node
I20260619 12:07:40.508526 16158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:40.508869 16158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:40.510075 16158 hybrid_clock.cc:648] HybridClock initialized: now 1781870860510035 us; error 55 us; skew 500 ppm
Jun 19 12:07:40 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:40.512857 16158 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:07:40.513410 16158 webserver.cc:460] Webserver started at http://127.0.203.190:39079/ using document root <none> and password file <none>
I20260619 12:07:40.513636 16158 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:40.513689 16158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:40.513839 16158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:40.514966 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "f607b569dbc0410084a04bab0782ed86"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "94258fbf97a4beb238d8229450454018"
server_key_iv: "5ecfeb30fd914a24e3731e83fa7f79c5"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.515455 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "f607b569dbc0410084a04bab0782ed86"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "94258fbf97a4beb238d8229450454018"
server_key_iv: "5ecfeb30fd914a24e3731e83fa7f79c5"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.516856 16158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:40.517846 16173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:40.518060 16158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:07:40.518180 16158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "f607b569dbc0410084a04bab0782ed86"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "94258fbf97a4beb238d8229450454018"
server_key_iv: "5ecfeb30fd914a24e3731e83fa7f79c5"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.518260 16158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:40.541052 16158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:40.543762 16158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:40.543901 16158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:40.549049 16158 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:45807
I20260619 12:07:40.549110 16225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:45807 every 8 connection(s)
I20260619 12:07:40.549511 16158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:07:40.550524 16226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:40.552985 16226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Bootstrap starting.
I20260619 12:07:40.553591 16226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:40.553885 16226 log.cc:824] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:40.554731 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 16158
I20260619 12:07:40.554838 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:07:40.554976 16226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: No bootstrap required, opened a new log
I20260619 12:07:40.555029 814 external_mini_cluster.cc:1428] Setting key be0fa595bd8e949812f208be7a6f6a32
I20260619 12:07:40.557049 16226 raft_consensus.cc:348] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:07:40.557243 16226 raft_consensus.cc:374] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:40.557281 16226 raft_consensus.cc:729] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f607b569dbc0410084a04bab0782ed86, State: Initialized, Role: FOLLOWER
I20260619 12:07:40.557472 16226 consensus_queue.cc:260] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:07:40.557555 16226 raft_consensus.cc:388] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:07:40.557578 16226 raft_consensus.cc:482] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:07:40.557614 16226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:40.558815 16226 raft_consensus.cc:504] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:07:40.558938 16226 leader_election.cc:302] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f607b569dbc0410084a04bab0782ed86; no voters:
I20260619 12:07:40.559141 16226 leader_election.cc:288] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:07:40.559221 16231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:40.559437 16226 sys_catalog.cc:564] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:07:40.559487 16231 raft_consensus.cc:686] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 LEADER]: Becoming Leader. State: Replica: f607b569dbc0410084a04bab0782ed86, State: Running, Role: LEADER
I20260619 12:07:40.559731 16231 consensus_queue.cc:237] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:07:40.560462 16232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f607b569dbc0410084a04bab0782ed86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } } }
I20260619 12:07:40.560570 16232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:40.560856 16233 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f607b569dbc0410084a04bab0782ed86. Latest consensus state: current_term: 1 leader_uuid: "f607b569dbc0410084a04bab0782ed86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } } }
I20260619 12:07:40.560983 16233 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:07:40.561161 16243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:07:40.561987 16243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 12:07:40 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:40.564445 16243 catalog_manager.cc:1347] Generated new cluster ID: 5d54e1d7263f4629a9e46af626a4c5cd
I20260619 12:07:40.564494 16243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:07:40.569360 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:40.555714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.0.203.190:45807)
0619 12:07:40.555994 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:40.556005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:40.556042 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:07:40.557009 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:40.557019 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:40.557089 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:40.557283 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:40.558044 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.559491 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.560713 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.561393 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.561519 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:40.564925 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:40.564931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:40.564931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:40.564947 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:40.567205 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:40.567774 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:40.567778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:40.567779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:40.567953 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:40.568249 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:40.568251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:40.568251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:40.568634 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:40.568817 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:40.568948 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 12:07:40.569123 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":141,"threads_started":1}
I20260619 12:07:40.578855 16243 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:07:40.579798 16243 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:07:40.586453 16243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Generated new TSK 0
I20260619 12:07:40.586704 16243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:07:40.618083 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:45807
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:40.699147 16252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:40.699414 16252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:40.699443 16252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:40.700660 16252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:40.700711 16252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:40.700740 16252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:07:40.702479 16252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:40.702699 16252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:40.702893 16252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:07:40.708352 16252 server_base.cc:1034] running on GCE node
W20260619 12:07:40.708328 16257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:40.708328 16260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:40.708315 16258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:40.708981 16252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:40.709260 16252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:40.710426 16252 hybrid_clock.cc:648] HybridClock initialized: now 1781870860710408 us; error 38 us; skew 500 ppm
Jun 19 12:07:40 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:40.713616 16252 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:07:40.714221 16252 webserver.cc:460] Webserver started at http://127.0.203.129:37343/ using document root <none> and password file <none>
I20260619 12:07:40.714459 16252 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:40.714512 16252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:40.714638 16252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:40.715938 16252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "0428b465bb6c423d8c39ac4a8191e5c1"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "26d7c80f85ee53af2ef36699a0433d63"
server_key_iv: "7421fe71eaa5314deee3a143ad8b2006"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.716301 16252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "0428b465bb6c423d8c39ac4a8191e5c1"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "26d7c80f85ee53af2ef36699a0433d63"
server_key_iv: "7421fe71eaa5314deee3a143ad8b2006"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.717831 16252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:07:40.718690 16267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:40.718928 16252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:40.719015 16252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "0428b465bb6c423d8c39ac4a8191e5c1"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "26d7c80f85ee53af2ef36699a0433d63"
server_key_iv: "7421fe71eaa5314deee3a143ad8b2006"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.719095 16252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:40.734484 16252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:40.736877 16252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:40.737013 16252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:40.737277 16252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:40.737624 16252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:40.737660 16252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:40.737692 16252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:40.737717 16252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:40.744670 16252 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:41991
I20260619 12:07:40.744789 16380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:41991 every 8 connection(s)
I20260619 12:07:40.745085 16252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:07:40 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:40.753275 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 16252
I20260619 12:07:40.753418 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:07:40.753635 814 external_mini_cluster.cc:1428] Setting key 0cfde225afc4798504d94cb38a691749
I20260619 12:07:40.755797 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:40.745959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:40911 (local address 127.0.203.190:45807)
0619 12:07:40.746010 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:40.746012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:40.746267 (+ 255us) server_negotiation.cc:407] Connection header received
0619 12:07:40.746816 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:40.746816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:40.746826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:40.746915 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:40.748011 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.748533 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.749640 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.749882 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.749963 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:40.752537 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:40.752540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:40.752540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:40.752554 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:40.754186 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:40.754621 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:40.754623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:40.754624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:40.754697 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:40.754934 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:40.754934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:40.754935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:40.755073 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:40.755134 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:40.755551 (+ 417us) server_negotiation.cc:299] Negotiation successful
0619 12:07:40.755689 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:07:40.755982 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:40.745928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:40911)
0619 12:07:40.746107 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:40.746163 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:40.746723 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:40.746981 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:40.746985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:40.747672 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:40.747920 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:40.747926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.748615 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.748616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:40.749516 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:40.749519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.749989 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.749989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:40.750111 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:40.750756 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:40.750766 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:40.752378 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:40.754276 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:40.754285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:40.754288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:40.754545 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:40.754752 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:40.754753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:40.754753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:40.754876 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:40.755127 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:40.755181 (+ 54us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:40.755346 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 12:07:40.755519 (+ 173us) client_negotiation.cc:240] Negotiation successful
0619 12:07:40.755722 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260619 12:07:40.756690 16381 heartbeater.cc:347] Connected to a master server at 127.0.203.190:45807
I20260619 12:07:40.756821 16381 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:40.757040 16381 heartbeater.cc:511] Master 127.0.203.190:45807 requested a full tablet report, sending...
I20260619 12:07:40.757611 16190 ts_manager.cc:194] Registered new tserver with Master: 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129:41991)
I20260619 12:07:40.758337 16190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:40911
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:07:40.782955 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:45807
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:40.873940 16386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:40.874218 16386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:40.874256 16386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:40.875685 16386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:40.875743 16386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:40.875782 16386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:07:40.877460 16386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:40.877823 16386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:40.878086 16386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:40.883690 16394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:40.883692 16391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:40.883697 16392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:40.884114 16386 server_base.cc:1034] running on GCE node
I20260619 12:07:40.884337 16386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:40.884606 16386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:40.885752 16386 hybrid_clock.cc:648] HybridClock initialized: now 1781870860885736 us; error 42 us; skew 500 ppm
Jun 19 12:07:40 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:40.888743 16386 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:07:40.889304 16386 webserver.cc:460] Webserver started at http://127.0.203.130:46575/ using document root <none> and password file <none>
I20260619 12:07:40.889565 16386 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:40.889616 16386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:40.889739 16386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:40.891014 16386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "fa7a0029147f4adea28b64769d6fd208"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "aa88339d398208a029c99fdfa77ca97e"
server_key_iv: "5b96641eebeb1cb0ead84bc7281db574"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.891428 16386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "fa7a0029147f4adea28b64769d6fd208"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "aa88339d398208a029c99fdfa77ca97e"
server_key_iv: "5b96641eebeb1cb0ead84bc7281db574"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.892992 16386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:40.893931 16401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:40.894204 16386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:40.894297 16386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "fa7a0029147f4adea28b64769d6fd208"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "aa88339d398208a029c99fdfa77ca97e"
server_key_iv: "5b96641eebeb1cb0ead84bc7281db574"
server_key_version: "encryptionkey@0"
I20260619 12:07:40.894383 16386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:40.910208 16386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:40.912767 16386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:40.912911 16386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:40.913215 16386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:40.913583 16386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:40.913622 16386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:40.913655 16386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:40.913672 16386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:40.921712 16386 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:34723
I20260619 12:07:40.921859 16514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:34723 every 8 connection(s)
I20260619 12:07:40.922241 16386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:07:40 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:40.929071 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 16386
I20260619 12:07:40.929179 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:07:40.929368 814 external_mini_cluster.cc:1428] Setting key 80a219b713a8228a03e3b5f58d568354
I20260619 12:07:40.932804 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:40.923039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:41493 (local address 127.0.203.190:45807)
0619 12:07:40.923093 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:40.923095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:40.923398 (+ 303us) server_negotiation.cc:407] Connection header received
0619 12:07:40.924020 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:40.924021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:40.924034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:40.924095 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:40.925145 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.925700 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.926717 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.926925 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.926982 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:40.929555 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:40.929558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:40.929559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:40.929570 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:40.931139 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:40.931725 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:40.931726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:40.931727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:40.931815 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:40.932068 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:40.932069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:40.932069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:40.932240 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:40.932311 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:40.932591 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 12:07:40.932706 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:07:40.932878 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:40.923011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:41493)
0619 12:07:40.923211 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:40.923266 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:40.923912 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:40.924184 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:40.924190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:40.924812 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:40.925050 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:40.925054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.925783 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.925784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:40.926640 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:40.926642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:40.926994 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:40.926995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:40.927127 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:40.927757 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:40.927767 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:40.929394 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:40.931252 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:40.931261 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:40.931264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:40.931632 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:40.931888 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:40.931888 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:40.931889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:40.932000 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:40.932296 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:40.932302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:40.932439 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 12:07:40.932527 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:07:40.932669 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260619 12:07:40.933490 16515 heartbeater.cc:347] Connected to a master server at 127.0.203.190:45807
I20260619 12:07:40.933593 16515 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:40.933782 16515 heartbeater.cc:511] Master 127.0.203.190:45807 requested a full tablet report, sending...
I20260619 12:07:40.934448 16190 ts_manager.cc:194] Registered new tserver with Master: fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723)
I20260619 12:07:40.934851 16190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:41493
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:07:40.959575 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:45807
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:07:41.050650 16520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:07:41.050935 16520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:07:41.050977 16520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:07:41.052416 16520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:07:41.052487 16520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:07:41.052533 16520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:07:41.054194 16520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:07:41.054549 16520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:07:41.054826 16520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:07:41.060355 16528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:41.060370 16526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:07:41.060328 16525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:07:41.060755 16520 server_base.cc:1034] running on GCE node
I20260619 12:07:41.061030 16520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:07:41.061314 16520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:07:41.062512 16520 hybrid_clock.cc:648] HybridClock initialized: now 1781870861062490 us; error 44 us; skew 500 ppm
Jun 19 12:07:41 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870861, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:07:41.065656 16520 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:07:41.066161 16520 webserver.cc:460] Webserver started at http://127.0.203.131:33009/ using document root <none> and password file <none>
I20260619 12:07:41.066399 16520 fs_manager.cc:359] Metadata directory not provided
I20260619 12:07:41.066449 16520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:07:41.066587 16520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:07:41.067751 16520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "d30fe60abe2d448bbaaa3cbcd1d06033"
format_stamp: "Formatted at 2026-06-19 12:07:41 on dist-test-slave-44vj"
server_key: "614e0ecb1d27b7da277a8ae28eed0aa1"
server_key_iv: "3dbd2939e2f1daec9096f3ea486ad80e"
server_key_version: "encryptionkey@0"
I20260619 12:07:41.068100 16520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "d30fe60abe2d448bbaaa3cbcd1d06033"
format_stamp: "Formatted at 2026-06-19 12:07:41 on dist-test-slave-44vj"
server_key: "614e0ecb1d27b7da277a8ae28eed0aa1"
server_key_iv: "3dbd2939e2f1daec9096f3ea486ad80e"
server_key_version: "encryptionkey@0"
I20260619 12:07:41.069509 16520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 12:07:41.070362 16535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:41.070566 16520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:07:41.070662 16520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "d30fe60abe2d448bbaaa3cbcd1d06033"
format_stamp: "Formatted at 2026-06-19 12:07:41 on dist-test-slave-44vj"
server_key: "614e0ecb1d27b7da277a8ae28eed0aa1"
server_key_iv: "3dbd2939e2f1daec9096f3ea486ad80e"
server_key_version: "encryptionkey@0"
I20260619 12:07:41.070734 16520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:07:41.083520 16520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:07:41.085558 16520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:07:41.085700 16520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:07:41.085948 16520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:07:41.086304 16520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:07:41.086359 16520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:41.086390 16520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:07:41.086406 16520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:07:41.092988 16520 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:44917
I20260619 12:07:41.093039 16648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:44917 every 8 connection(s)
I20260619 12:07:41.093374 16520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:07:41.095664 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 16520
I20260619 12:07:41.095765 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:07:41.095940 814 external_mini_cluster.cc:1428] Setting key 4b6424e1370d9df00d50a0c8a4c7208b
Jun 19 12:07:41 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870861, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:07:41.103628 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.094236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:47549 (local address 127.0.203.190:45807)
0619 12:07:41.094293 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.094295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.094571 (+ 276us) server_negotiation.cc:407] Connection header received
0619 12:07:41.095093 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.095095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.095107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.095175 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:41.096295 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.097081 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.098188 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.098384 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.098440 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.100793 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:41.100795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:41.100795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:41.100803 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:41.102218 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:41.102632 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:41.102633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:41.102633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:41.102691 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:41.102915 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:41.102915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:41.102916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:41.103066 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:41.103125 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.103457 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.103546 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:07:41.103754 16651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.094201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:47549)
0619 12:07:41.094410 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:41.094460 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:41.094992 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:41.095236 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:41.095240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:41.095926 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:07:41.096194 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.096202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.097175 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.097176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.098109 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.098112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.098455 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.098456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.098554 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.099118 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:07:41.099126 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:07:41.100678 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:07:41.102303 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:41.102309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:41.102312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:41.102563 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:41.102747 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:07:41.102747 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:07:41.102748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:07:41.102848 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:07:41.103126 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:07:41.103129 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:07:41.103264 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 12:07:41.103355 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:07:41.103548 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260619 12:07:41.104380 16649 heartbeater.cc:347] Connected to a master server at 127.0.203.190:45807
I20260619 12:07:41.104569 16649 heartbeater.cc:464] Registering TS with master...
I20260619 12:07:41.104858 16649 heartbeater.cc:511] Master 127.0.203.190:45807 requested a full tablet report, sending...
I20260619 12:07:41.105336 16190 ts_manager.cc:194] Registered new tserver with Master: d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
I20260619 12:07:41.105782 16190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:47549
I20260619 12:07:41.107962 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:07:41.116315 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.109664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.0.203.190:45807)
0619 12:07:41.109742 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.109744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.109882 (+ 138us) server_negotiation.cc:407] Connection header received
0619 12:07:41.109912 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.109913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.109923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.110012 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:07:41.110652 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.111190 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.112064 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.112264 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.112327 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.113544 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:07:41.113546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:07:41.113546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:07:41.113553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:07:41.114911 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:41.115346 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:41.115346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:41.115347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:41.115432 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:07:41.115751 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:07:41.115751 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:07:41.115752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:07:41.115925 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:07:41.116030 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.116172 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.116236 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:07:41.118388 16190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:07:41.120254 16190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:07:41.128255 16663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.123552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.0.203.130:34723)
0619 12:07:41.123958 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.123963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.123974 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:07:41.124018 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.124020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.124036 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.124110 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:41.124356 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.125222 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.126764 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.127498 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.127782 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.127876 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.128083 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.128142 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":97,"threads_started":1}
I20260619 12:07:41.128294 16665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.123255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34723 (local address 127.0.0.1:40642)
0619 12:07:41.123810 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:41.123823 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:41.123889 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:41.124195 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:41.124196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:41.124199 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:41.124271 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.124274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.125319 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.125319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.126594 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.126597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.127799 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.127801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.127965 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.127981 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 12:07:41.128055 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:07:41.128142 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":511,"threads_started":1}
I20260619 12:07:41.128422 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.123253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44917 (local address 127.0.0.1:52300)
0619 12:07:41.123580 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:41.123615 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:41.123697 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:41.124589 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:41.124589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:41.124592 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:41.124647 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.124649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.125855 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.125857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.126858 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.126861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.127999 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.128000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.128080 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.128088 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:07:41.128137 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:07:41.128199 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":181,"threads_started":1}
I20260619 12:07:41.128516 16664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.123624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.0.203.131:44917)
0619 12:07:41.124421 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.124425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.124436 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:07:41.124473 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.124475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.124489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.124574 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:41.124731 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.125742 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.126985 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.127819 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.127895 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.127959 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.128343 (+ 384us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.128405 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":79,"threads_started":1}
I20260619 12:07:41.128619 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.123250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41991 (local address 127.0.0.1:49864)
0619 12:07:41.123570 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:41.123615 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:41.123697 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:41.124067 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:41.124071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:41.124075 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:41.124175 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.124178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.125631 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.125632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.126933 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.126935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.128012 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.128012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.128080 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.128089 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:07:41.128137 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:07:41.128553 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":2,"thread_start_us":169,"threads_started":1}
I20260619 12:07:41.128911 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.123398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.0.203.129:41991)
0619 12:07:41.123887 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.123892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.123903 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:07:41.123948 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.123951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.123968 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.124056 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:41.124287 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.125497 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.127042 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.127819 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.127895 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.127959 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.128728 (+ 769us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.128805 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":97,"threads_started":1}
I20260619 12:07:41.129662 16583 tablet_service.cc:1467] Processing CreateTablet for tablet b0a07bed63ec454fa6dadab3ad36207e (DEFAULT_TABLE table=test-table [id=24720b39a0044fe7859eaf077e097146]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:41.129667 16449 tablet_service.cc:1467] Processing CreateTablet for tablet b0a07bed63ec454fa6dadab3ad36207e (DEFAULT_TABLE table=test-table [id=24720b39a0044fe7859eaf077e097146]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:41.129669 16315 tablet_service.cc:1467] Processing CreateTablet for tablet b0a07bed63ec454fa6dadab3ad36207e (DEFAULT_TABLE table=test-table [id=24720b39a0044fe7859eaf077e097146]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:07:41.130105 16315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a07bed63ec454fa6dadab3ad36207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:41.130127 16449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a07bed63ec454fa6dadab3ad36207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:41.130141 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a07bed63ec454fa6dadab3ad36207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:07:41.133838 16666 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Bootstrap starting.
I20260619 12:07:41.134446 16667 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Bootstrap starting.
I20260619 12:07:41.134732 16666 tablet_bootstrap.cc:654] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:41.135248 16667 tablet_bootstrap.cc:654] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:41.135486 16666 log.cc:824] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:41.135552 16668 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap starting.
I20260619 12:07:41.135869 16667 log.cc:824] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:41.136301 16668 tablet_bootstrap.cc:654] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Neither blocks nor log segments found. Creating new log.
I20260619 12:07:41.136332 16666 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: No bootstrap required, opened a new log
I20260619 12:07:41.136404 16666 ts_tablet_manager.cc:1397] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 12:07:41.136854 16668 log.cc:824] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:07:41.136902 16667 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: No bootstrap required, opened a new log
I20260619 12:07:41.136960 16667 ts_tablet_manager.cc:1397] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:41.137904 16668 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: No bootstrap required, opened a new log
I20260619 12:07:41.138019 16668 ts_tablet_manager.cc:1397] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:41.138278 16666 raft_consensus.cc:348] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.138448 16666 raft_consensus.cc:374] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:41.138482 16666 raft_consensus.cc:729] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d30fe60abe2d448bbaaa3cbcd1d06033, State: Initialized, Role: FOLLOWER
I20260619 12:07:41.138665 16666 consensus_queue.cc:260] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.138971 16649 heartbeater.cc:503] Master 127.0.203.190:45807 was elected leader, sending a full tablet report...
I20260619 12:07:41.138969 16666 ts_tablet_manager.cc:1428] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:07:41.138890 16667 raft_consensus.cc:348] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.139070 16667 raft_consensus.cc:374] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:41.139109 16667 raft_consensus.cc:729] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa7a0029147f4adea28b64769d6fd208, State: Initialized, Role: FOLLOWER
I20260619 12:07:41.139246 16667 consensus_queue.cc:260] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.139655 16667 ts_tablet_manager.cc:1428] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:07:41.139814 16515 heartbeater.cc:503] Master 127.0.203.190:45807 was elected leader, sending a full tablet report...
I20260619 12:07:41.140136 16668 raft_consensus.cc:348] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.140321 16668 raft_consensus.cc:374] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:07:41.140355 16668 raft_consensus.cc:729] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0428b465bb6c423d8c39ac4a8191e5c1, State: Initialized, Role: FOLLOWER
I20260619 12:07:41.140508 16668 consensus_queue.cc:260] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.140791 16668 ts_tablet_manager.cc:1428] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 12:07:41.140848 16381 heartbeater.cc:503] Master 127.0.203.190:45807 was elected leader, sending a full tablet report...
W20260619 12:07:41.172889 16516 tablet.cc:2369] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:07:41.245978 16382 tablet.cc:2369] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:07:41.344056 16650 tablet.cc:2369] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:07:41.543815 16673 raft_consensus.cc:482] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:07:41.543973 16673 raft_consensus.cc:504] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.544459 16673 leader_election.cc:288] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917), 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129:41991)
I20260619 12:07:41.548884 16664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.544812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:33567 (local address 127.0.203.131:44917)
0619 12:07:41.544877 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.544880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.545013 (+ 133us) server_negotiation.cc:407] Connection header received
0619 12:07:41.545091 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.545093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.545103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.545153 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:41.545295 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.546122 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.547804 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.548418 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.548502 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.548516 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.548743 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.548791 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:07:41.548884 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.544784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:42117 (local address 127.0.203.129:41991)
0619 12:07:41.544893 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.544895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.544932 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:07:41.545037 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.545039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.545051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.545121 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:07:41.545303 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.546103 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.547804 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.548406 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.548556 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.548569 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.548743 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.548792 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 12:07:41.548928 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.544712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44917 (local address 127.0.203.130:33567)
0619 12:07:41.544936 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:41.544946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:41.545012 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:41.545167 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:41.545168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:41.545169 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:41.545226 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.545229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.546240 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.546241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.547683 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.547686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.548542 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.548544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.548648 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.548654 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:07:41.548730 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:07:41.548786 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":41,"thread_start_us":97,"threads_started":1}
I20260619 12:07:41.548928 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.544682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41991 (local address 127.0.203.130:42117)
0619 12:07:41.544861 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:41.544873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:41.544946 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:41.545132 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:07:41.545133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:07:41.545136 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:07:41.545226 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.545229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.546240 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.546241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.547683 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:07:41.547686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.548569 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.548570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:07:41.548648 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.548655 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:07:41.548730 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:07:41.548781 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":109,"thread_start_us":89,"threads_started":1}
I20260619 12:07:41.549403 16603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a07bed63ec454fa6dadab3ad36207e" candidate_uuid: "fa7a0029147f4adea28b64769d6fd208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" is_pre_election: true
I20260619 12:07:41.549605 16603 raft_consensus.cc:2454] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa7a0029147f4adea28b64769d6fd208 in term 0.
I20260619 12:07:41.549575 16335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a07bed63ec454fa6dadab3ad36207e" candidate_uuid: "fa7a0029147f4adea28b64769d6fd208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" is_pre_election: true
I20260619 12:07:41.549707 16335 raft_consensus.cc:2454] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa7a0029147f4adea28b64769d6fd208 in term 0.
I20260619 12:07:41.549999 16402 leader_election.cc:302] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d30fe60abe2d448bbaaa3cbcd1d06033, fa7a0029147f4adea28b64769d6fd208; no voters:
I20260619 12:07:41.550186 16673 raft_consensus.cc:2781] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:07:41.550235 16673 raft_consensus.cc:482] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:07:41.550259 16673 raft_consensus.cc:3037] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:41.550990 16673 raft_consensus.cc:504] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.551132 16673 leader_election.cc:288] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [CANDIDATE]: Term 1 election: Requested vote from peers d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917), 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129:41991)
I20260619 12:07:41.551384 16603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a07bed63ec454fa6dadab3ad36207e" candidate_uuid: "fa7a0029147f4adea28b64769d6fd208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033"
I20260619 12:07:41.551337 16335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a07bed63ec454fa6dadab3ad36207e" candidate_uuid: "fa7a0029147f4adea28b64769d6fd208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0428b465bb6c423d8c39ac4a8191e5c1"
I20260619 12:07:41.551469 16603 raft_consensus.cc:3037] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:41.551469 16335 raft_consensus.cc:3037] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:07:41.552376 16335 raft_consensus.cc:2454] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa7a0029147f4adea28b64769d6fd208 in term 1.
I20260619 12:07:41.552376 16603 raft_consensus.cc:2454] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa7a0029147f4adea28b64769d6fd208 in term 1.
I20260619 12:07:41.552650 16403 leader_election.cc:302] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1, fa7a0029147f4adea28b64769d6fd208; no voters:
I20260619 12:07:41.552762 16673 raft_consensus.cc:2781] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:07:41.552999 16673 raft_consensus.cc:686] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 1 LEADER]: Becoming Leader. State: Replica: fa7a0029147f4adea28b64769d6fd208, State: Running, Role: LEADER
I20260619 12:07:41.553125 16673 consensus_queue.cc:237] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:07:41.554117 16190 catalog_manager.cc:5495] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa7a0029147f4adea28b64769d6fd208 (127.0.203.130). New cstate: current_term: 1 leader_uuid: "fa7a0029147f4adea28b64769d6fd208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } health_report { overall_health: HEALTHY } } }
I20260619 12:07:41.572103 16663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:07:41.569813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.0.203.130:34723)
0619 12:07:41.569882 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:07:41.569884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:07:41.569892 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:07:41.569935 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:07:41.569936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:07:41.569963 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:07:41.570029 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:07:41.570166 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.570640 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.571393 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:07:41.571579 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:07:41.571630 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:07:41.571795 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:07:41.571907 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:07:41.571983 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:07:41.572000 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 12:07:41.572021 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:07:41.574201 16335 raft_consensus.cc:1261] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Refusing update from remote peer fa7a0029147f4adea28b64769d6fd208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:41.574244 16603 raft_consensus.cc:1261] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Refusing update from remote peer fa7a0029147f4adea28b64769d6fd208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:07:41.574575 16677 consensus_queue.cc:1035] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:41.574738 16673 consensus_queue.cc:1035] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:07:41.577634 16682 mvcc.cc:205] Tried to move back new op lower bound from 7298543049004949504 to 7298543048922226688. Current Snapshot: MvccSnapshot[applied={T|T < 7298543049004949504}]
I20260619 12:07:41.580215 16190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:41.580358 16190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:07:41.581246 16190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Sending DeleteTablet for 3 replicas of tablet b0a07bed63ec454fa6dadab3ad36207e
I20260619 12:07:41.581626 16315 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a07bed63ec454fa6dadab3ad36207e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:41 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:49864
I20260619 12:07:41.581697 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 16252
I20260619 12:07:41.581700 16583 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a07bed63ec454fa6dadab3ad36207e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:41 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:52300
I20260619 12:07:41.583775 16449 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a07bed63ec454fa6dadab3ad36207e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:41 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:40642
I20260619 12:07:41.584069 16694 tablet_replica.cc:330] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: stopping tablet replica
I20260619 12:07:41.584208 16694 raft_consensus.cc:2229] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:07:41.584383 16694 raft_consensus.cc:2258] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:41.584884 16694 ts_tablet_manager.cc:1905] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:07:41.586803 16694 ts_tablet_manager.cc:1918] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:41.586918 16694 log.cc:1196] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/b0a07bed63ec454fa6dadab3ad36207e
I20260619 12:07:41.587347 16694 ts_tablet_manager.cc:1939] T b0a07bed63ec454fa6dadab3ad36207e P fa7a0029147f4adea28b64769d6fd208: Deleting consensus metadata
I20260619 12:07:41.587679 16693 tablet_replica.cc:330] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: stopping tablet replica
I20260619 12:07:41.587842 16693 raft_consensus.cc:2229] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:07:41.588006 16693 raft_consensus.cc:2258] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:07:41.588176 16176 catalog_manager.cc:4841] TS fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723): tablet b0a07bed63ec454fa6dadab3ad36207e (table test-table [id=24720b39a0044fe7859eaf077e097146]) successfully deleted
W20260619 12:07:41.588382 16175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:41991 (error 108)
I20260619 12:07:41.588591 16693 ts_tablet_manager.cc:1905] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:07:41.589386 16660 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:07:41.589055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41991 (local address 127.0.0.1:49880)
0619 12:07:41.589135 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 12:07:41.589151 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:07:41.589212 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:07:41.589327 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:41991: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:07:41.589696 16175 catalog_manager.cc:4584] TS 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129:41991): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0a07bed63ec454fa6dadab3ad36207e: Network error: Client connection negotiation failed: client connection to 127.0.203.129:41991: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:07:41.589807 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 16386
I20260619 12:07:41.590975 16693 ts_tablet_manager.cc:1918] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:07:41.591037 16693 log.cc:1196] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/b0a07bed63ec454fa6dadab3ad36207e
I20260619 12:07:41.591207 16693 ts_tablet_manager.cc:1939] T b0a07bed63ec454fa6dadab3ad36207e P d30fe60abe2d448bbaaa3cbcd1d06033: Deleting consensus metadata
I20260619 12:07:41.591645 16174 catalog_manager.cc:4841] TS d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917): tablet b0a07bed63ec454fa6dadab3ad36207e (table test-table [id=24720b39a0044fe7859eaf077e097146]) successfully deleted
I20260619 12:07:41.595819 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 16520
I20260619 12:07:41.601359 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 16158
I20260619 12:08:01.607592 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:45807
--webserver_interface=127.0.203.190
--webserver_port=39079
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:01.696573 16696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:01.696825 16696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:01.696859 16696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:01.698203 16696 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:08:01.698257 16696 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:08:01.698274 16696 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:08:01.698289 16696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:01.698307 16696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:01.698323 16696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:01.698341 16696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:01.698360 16696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:01.700171 16696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:45807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=39079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:01.700457 16696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:01.700683 16696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:01.710444 16702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:01.710577 16703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:01.711038 16696 server_base.cc:1034] running on GCE node
W20260619 12:08:01.711079 16705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:01.711462 16696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:01.711884 16696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:01.713089 16696 hybrid_clock.cc:648] HybridClock initialized: now 1781870881713068 us; error 42 us; skew 500 ppm
Jun 19 12:08:01 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:01.716351 16696 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:08:01.716998 16696 webserver.cc:460] Webserver started at http://127.0.203.190:39079/ using document root <none> and password file <none>
I20260619 12:08:01.717280 16696 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:01.717351 16696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:01.719113 16696 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:08:01.720058 16712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:01.720353 16696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:08:01.720463 16696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "f607b569dbc0410084a04bab0782ed86"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "94258fbf97a4beb238d8229450454018"
server_key_iv: "5ecfeb30fd914a24e3731e83fa7f79c5"
server_key_version: "encryptionkey@0"
I20260619 12:08:01.720980 16696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:01.733870 16696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:01.736579 16696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:01.736758 16696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:01.741714 16696 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:45807
I20260619 12:08:01.741778 16764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:45807 every 8 connection(s)
I20260619 12:08:01.742244 16696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:01.744128 16765 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:08:01.744447 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 16696
I20260619 12:08:01.744796 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:41991
--local_ip_for_outbound_sockets=127.0.203.129
--tserver_master_addrs=127.0.203.190:45807
--webserver_port=37343
--webserver_interface=127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:08:01.745072 16765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Bootstrap starting.
I20260619 12:08:01.753897 16765 log.cc:824] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:01.756340 16765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:08:01.756670 16765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Bootstrap complete.
I20260619 12:08:01.759083 16765 raft_consensus.cc:348] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:01.759351 16765 raft_consensus.cc:729] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f607b569dbc0410084a04bab0782ed86, State: Initialized, Role: FOLLOWER
I20260619 12:08:01.759557 16765 consensus_queue.cc:260] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:01.759626 16765 raft_consensus.cc:388] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:01.759682 16765 raft_consensus.cc:482] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:01.759745 16765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:08:01.760881 16765 raft_consensus.cc:504] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:01.760982 16765 leader_election.cc:302] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f607b569dbc0410084a04bab0782ed86; no voters:
I20260619 12:08:01.761174 16765 leader_election.cc:288] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:08:01.761224 16769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:08:01.761435 16769 raft_consensus.cc:686] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 LEADER]: Becoming Leader. State: Replica: f607b569dbc0410084a04bab0782ed86, State: Running, Role: LEADER
I20260619 12:08:01.761482 16765 sys_catalog.cc:564] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:01.761648 16769 consensus_queue.cc:237] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:01.762018 16771 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f607b569dbc0410084a04bab0782ed86. Latest consensus state: current_term: 2 leader_uuid: "f607b569dbc0410084a04bab0782ed86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } } }
I20260619 12:08:01.762100 16771 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:01.762605 16770 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f607b569dbc0410084a04bab0782ed86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } } }
I20260619 12:08:01.762706 16770 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:01.762830 16775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:01.763742 16775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:01.763847 16775 catalog_manager.cc:1259] Loaded cluster ID: 5d54e1d7263f4629a9e46af626a4c5cd
I20260619 12:08:01.763883 16775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:01.765067 16775 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:01.765270 16775 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Loaded TSK: 0
I20260619 12:08:01.776722 16775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Generated new TSK 1
I20260619 12:08:01.776896 16775 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:08:01.839756 16767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:01.840055 16767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:01.840093 16767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:01.841588 16767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:01.841657 16767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:01.841701 16767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:01.843530 16767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:41991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=37343
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:01.843844 16767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:01.844112 16767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:01.850183 16793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:01.850183 16795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:01.850418 16792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:01.850853 16767 server_base.cc:1034] running on GCE node
I20260619 12:08:01.851187 16767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:01.851611 16767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:01.852823 16767 hybrid_clock.cc:648] HybridClock initialized: now 1781870881852799 us; error 38 us; skew 500 ppm
Jun 19 12:08:01 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:01.856158 16767 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:08:01.856818 16767 webserver.cc:460] Webserver started at http://127.0.203.129:37343/ using document root <none> and password file <none>
I20260619 12:08:01.857097 16767 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:01.857183 16767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:01.859092 16767 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 12:08:01.860128 16802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:01.860457 16767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:08:01.860560 16767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "0428b465bb6c423d8c39ac4a8191e5c1"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "26d7c80f85ee53af2ef36699a0433d63"
server_key_iv: "7421fe71eaa5314deee3a143ad8b2006"
server_key_version: "encryptionkey@0"
I20260619 12:08:01.861035 16767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:01.873016 16767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:01.875959 16767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:01.876149 16767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:01.876590 16767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:01.877204 16809 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:08:01.880025 16767 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:08:01.880116 16767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260619 12:08:01.880163 16767 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:08:01.881026 16767 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:08:01.881122 16767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:01.881369 16809 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap starting.
I20260619 12:08:01.889312 16767 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:41991
I20260619 12:08:01.889429 16916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:41991 every 8 connection(s)
I20260619 12:08:01.889973 16767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:08:01.891387 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 16767
I20260619 12:08:01.891685 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:34723
--local_ip_for_outbound_sockets=127.0.203.130
--tserver_master_addrs=127.0.203.190:45807
--webserver_port=46575
--webserver_interface=127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:08:01.893435 16809 log.cc:824] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:01.896719 16809 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:08:01.897130 16809 tablet_bootstrap.cc:492] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap complete.
I20260619 12:08:01.898406 16809 ts_tablet_manager.cc:1397] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20260619 12:08:01.900622 16809 raft_consensus.cc:348] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:08:01.901126 16809 raft_consensus.cc:729] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0428b465bb6c423d8c39ac4a8191e5c1, State: Initialized, Role: FOLLOWER
I20260619 12:08:01.901352 16809 consensus_queue.cc:260] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } } peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } }
I20260619 12:08:01.901793 16809 ts_tablet_manager.cc:1428] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
Jun 19 12:08:01 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:01.910248 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:01.894584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:34915 (local address 127.0.203.190:45807)
0619 12:08:01.897171 (+ 2587us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:01.897185 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:01.898498 (+ 1313us) server_negotiation.cc:407] Connection header received
0619 12:08:01.898739 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:01.898746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:01.898823 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:01.898981 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:01.900501 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.902210 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.903428 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.903892 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.904011 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:01.906524 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:01.906529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:01.906529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:01.906542 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:01.908306 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:01.908764 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:01.908769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:01.908769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:01.908868 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:01.909205 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:01.909206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:01.909207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:01.909470 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:01.909557 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:01.909855 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 12:08:01.910007 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2484,"thread_start_us":103,"threads_started":1}
I20260619 12:08:01.910248 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:01.894464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:34915)
0619 12:08:01.897317 (+ 2853us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:01.897375 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:01.898236 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:01.899157 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:01.899162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:01.899796 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:01.900292 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:01.900304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.902340 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.902342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:01.903279 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:01.903284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.903993 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.903995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:01.904129 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:01.904731 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:01.904741 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:01.906342 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:01.908402 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:01.908412 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:01.908415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:01.908687 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:01.908934 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:01.908935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:01.908935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:01.909144 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:01.909545 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:01.909551 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:01.909696 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 12:08:01.909793 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:08:01.910012 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2801,"thread_start_us":106,"threads_started":1}
I20260619 12:08:01.911432 16917 heartbeater.cc:347] Connected to a master server at 127.0.203.190:45807
I20260619 12:08:01.911616 16917 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:01.911861 16917 heartbeater.cc:511] Master 127.0.203.190:45807 requested a full tablet report, sending...
I20260619 12:08:01.912839 16729 ts_manager.cc:194] Registered new tserver with Master: 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129:41991)
I20260619 12:08:01.914042 16729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:34915
I20260619 12:08:01.915266 16917 heartbeater.cc:503] Master 127.0.203.190:45807 was elected leader, sending a full tablet report...
I20260619 12:08:01.923049 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:01.913869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.0.203.129:41991)
0619 12:08:01.919852 (+ 5983us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:01.919857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:01.919867 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:08:01.919906 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:01.919908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:01.919925 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:01.919998 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:01.920220 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.920980 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.922093 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.922615 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.922685 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:01.922754 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:01.922889 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 12:08:01.922940 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5943,"thread_start_us":89,"threads_started":1}
I20260619 12:08:01.923049 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:01.913547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41991 (local address 127.0.0.1:42192)
0619 12:08:01.919526 (+ 5979us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:01.919585 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:01.919697 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:01.920022 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:01.920026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:01.920030 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:01.920143 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:01.920146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.921083 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.921085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:01.922004 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:01.922008 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:01.922696 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:01.922697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:01.922785 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:01.922802 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:08:01.922869 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 12:08:01.922951 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5930,"thread_start_us":127,"threads_started":1}
I20260619 12:08:01.923913 16851 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a07bed63ec454fa6dadab3ad36207e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:07:41 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42192
I20260619 12:08:01.924676 16926 tablet_replica.cc:330] stopping tablet replica
I20260619 12:08:01.924854 16926 raft_consensus.cc:2229] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:01.924939 16926 raft_consensus.cc:2258] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:01.925333 16926 ts_tablet_manager.cc:1905] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:01.928027 16926 ts_tablet_manager.cc:1918] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:01.928143 16926 log.cc:1196] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal/wals/b0a07bed63ec454fa6dadab3ad36207e
I20260619 12:08:01.928365 16926 ts_tablet_manager.cc:1939] T b0a07bed63ec454fa6dadab3ad36207e P 0428b465bb6c423d8c39ac4a8191e5c1: Deleting consensus metadata
I20260619 12:08:01.929157 16714 catalog_manager.cc:4841] TS 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129:41991): tablet b0a07bed63ec454fa6dadab3ad36207e (table test-table [id=24720b39a0044fe7859eaf077e097146]) successfully deleted
W20260619 12:08:01.996860 16919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:01.997107 16919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:01.997133 16919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:01.998417 16919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:01.998477 16919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:01.998507 16919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:02.001408 16919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:34723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=46575
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:02.001724 16919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:02.001942 16919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:08:02.008119 16919 server_base.cc:1034] running on GCE node
W20260619 12:08:02.008036 16931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:02.008036 16932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:02.008060 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:02.008867 16919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:02.009157 16919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:02.010385 16919 hybrid_clock.cc:648] HybridClock initialized: now 1781870882010340 us; error 65 us; skew 500 ppm
Jun 19 12:08:02 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:02.014014 16919 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:08:02.015311 16919 webserver.cc:460] Webserver started at http://127.0.203.130:46575/ using document root <none> and password file <none>
I20260619 12:08:02.015765 16919 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:02.015867 16919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:02.017715 16919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:08:02.018664 16941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:02.019021 16919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:02.019146 16919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "fa7a0029147f4adea28b64769d6fd208"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "aa88339d398208a029c99fdfa77ca97e"
server_key_iv: "5b96641eebeb1cb0ead84bc7281db574"
server_key_version: "encryptionkey@0"
I20260619 12:08:02.019713 16919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:02.045094 16919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:02.047849 16919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:02.048012 16919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:02.048259 16919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:02.048600 16919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:02.048636 16919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:02.048667 16919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:02.048684 16919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:02.055234 16919 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:34723
I20260619 12:08:02.055346 17054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:34723 every 8 connection(s)
I20260619 12:08:02.055904 16919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:02.061602 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 16919
I20260619 12:08:02.061936 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:44917
--local_ip_for_outbound_sockets=127.0.203.131
--tserver_master_addrs=127.0.203.190:45807
--webserver_port=33009
--webserver_interface=127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:08:02 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:02.067229 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.056935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:33051 (local address 127.0.203.190:45807)
0619 12:08:02.057006 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.057008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.057299 (+ 291us) server_negotiation.cc:407] Connection header received
0619 12:08:02.057946 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.057948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.057963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.058020 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:02.059222 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.059843 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.060959 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.061190 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.061250 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.064248 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:02.064250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:02.064250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:02.064260 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:02.065727 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.066182 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.066183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.066183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.066243 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.066481 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.066482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.066482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.066622 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:02.066686 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.066992 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.067103 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:08:02.067456 17057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.056926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:33051)
0619 12:08:02.057132 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:02.057176 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:02.057804 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:02.058105 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:02.058110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:02.058853 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:02.059104 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.059109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.059946 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.059947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.060871 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.060874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.061266 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.061267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.061410 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.062155 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:02.062170 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:02.064141 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:02.065825 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:02.065833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:02.065837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:02.066112 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:02.066302 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:02.066303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:02.066303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:02.066425 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:02.066681 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:02.066689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:02.066865 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 12:08:02.066966 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:08:02.067164 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":115,"threads_started":1}
I20260619 12:08:02.068188 17055 heartbeater.cc:347] Connected to a master server at 127.0.203.190:45807
I20260619 12:08:02.068295 17055 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:02.068596 17055 heartbeater.cc:511] Master 127.0.203.190:45807 requested a full tablet report, sending...
I20260619 12:08:02.069193 16729 ts_manager.cc:194] Registered new tserver with Master: fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723)
I20260619 12:08:02.069793 16729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:33051
W20260619 12:08:02.160355 17058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:02.160594 17058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:02.160625 17058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:02.162011 17058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:02.162070 17058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:02.162106 17058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:02.163777 17058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:44917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=33009
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:02.164047 17058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:02.164270 17058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:08:02.170228 17058 server_base.cc:1034] running on GCE node
W20260619 12:08:02.170152 17063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:02.170181 17066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:02.170181 17064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:02.170805 17058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:02.171115 17058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:02.172334 17058 hybrid_clock.cc:648] HybridClock initialized: now 1781870882172307 us; error 50 us; skew 500 ppm
Jun 19 12:08:02 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:02.175752 17058 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:08:02.176599 17058 webserver.cc:460] Webserver started at http://127.0.203.131:33009/ using document root <none> and password file <none>
I20260619 12:08:02.176924 17058 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:02.177000 17058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:02.178625 17058 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:08:02.179416 17073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:02.179656 17058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:02.179768 17058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "d30fe60abe2d448bbaaa3cbcd1d06033"
format_stamp: "Formatted at 2026-06-19 12:07:41 on dist-test-slave-44vj"
server_key: "614e0ecb1d27b7da277a8ae28eed0aa1"
server_key_iv: "3dbd2939e2f1daec9096f3ea486ad80e"
server_key_version: "encryptionkey@0"
I20260619 12:08:02.180178 17058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:02.198931 17058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:02.201236 17058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:02.201397 17058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:02.201648 17058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:02.201968 17058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:02.202004 17058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:02.202034 17058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:02.202049 17058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:02.209025 17058 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:44917
I20260619 12:08:02.209096 17186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:44917 every 8 connection(s)
I20260619 12:08:02.209470 17058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:08:02 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:02.218282 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 17058
I20260619 12:08:02.221741 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.210283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:47797 (local address 127.0.203.190:45807)
0619 12:08:02.210344 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.210346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.210561 (+ 215us) server_negotiation.cc:407] Connection header received
0619 12:08:02.211147 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.211149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.211167 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.211273 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:02.212760 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.213351 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.214628 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.214872 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.214955 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.217782 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:02.217789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:02.217789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:02.217803 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:02.219700 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.220282 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.220283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.220284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.220428 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.220846 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.220847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.220848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.220982 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:02.221038 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.221539 (+ 501us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.221635 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:08:02.222008 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.210243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:47797)
0619 12:08:02.210411 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:02.210458 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:02.210999 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:02.211546 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:02.211553 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:02.212314 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:02.212599 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.212605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.213540 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.213542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.214472 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.214477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.215019 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.215020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.215183 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.215827 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:02.215836 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:02.217612 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:02.219805 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:02.219820 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:02.219824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:02.220207 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:02.220553 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:02.220555 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:02.220555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:02.220766 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:02.221091 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:02.221104 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:02.221331 (+ 227us) client_negotiation.cc:769] Sending connection context
0619 12:08:02.221491 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 12:08:02.221728 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260619 12:08:02.222817 17187 heartbeater.cc:347] Connected to a master server at 127.0.203.190:45807
I20260619 12:08:02.223064 17187 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:02.223491 17187 heartbeater.cc:511] Master 127.0.203.190:45807 requested a full tablet report, sending...
I20260619 12:08:02.224152 16729 ts_manager.cc:194] Registered new tserver with Master: d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
I20260619 12:08:02.224611 16729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:47797
I20260619 12:08:02.227396 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.218884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.0.203.190:45807)
0619 12:08:02.219001 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.219003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.219131 (+ 128us) server_negotiation.cc:407] Connection header received
0619 12:08:02.219204 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.219206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.219224 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.219317 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:02.220098 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.220780 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.221979 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.222310 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.222389 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.224265 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:02.224267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:02.224267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:02.224277 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:02.225818 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.226205 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.226205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.226206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.226262 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.226658 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.226659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.226660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.226805 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:02.226975 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.227126 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.227229 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260619 12:08:02.228371 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:08:02.231540 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.228782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42708 (local address 127.0.203.190:45807)
0619 12:08:02.228848 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.228850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.228944 (+ 94us) server_negotiation.cc:407] Connection header received
0619 12:08:02.229029 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.229030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.229080 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.229145 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:02.229279 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.229817 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.230580 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.230808 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.230865 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.231020 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:02.231087 (+ 67us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 12:08:02.231464 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42708: token expired
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 12:08:02.231583 16654 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.203.190:45807)
W20260619 12:08:02.231702 17191 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42708: token expired
I20260619 12:08:02.239319 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.231967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.0.203.190:45807)
0619 12:08:02.232005 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.232007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.232014 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:08:02.232102 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.232104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.232115 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.232162 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:02.232842 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.233614 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.234876 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.235090 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.235149 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.236430 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:02.236431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:02.236432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:02.236440 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:02.238144 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.238511 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.238513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.238513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.238570 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:02.238731 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:02.238732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:02.238732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:02.238914 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:02.239017 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.239118 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.239208 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:08:02.240861 16729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:08:02.243324 16729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:08:02.246379 16851 tablet_service.cc:1467] Processing CreateTablet for tablet fa8c98e18d614328a35e325ac025f9b0 (DEFAULT_TABLE table=test-table [id=339f067a1b6c43aa9bc2bb2f5ae098f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:02.246615 16851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa8c98e18d614328a35e325ac025f9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:02.248383 16809 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap starting.
I20260619 12:08:02.248653 16809 tablet_bootstrap.cc:654] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:02.249150 16809 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: No bootstrap required, opened a new log
I20260619 12:08:02.249192 16809 ts_tablet_manager.cc:1397] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:08:02.249300 16809 raft_consensus.cc:348] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.249370 16809 raft_consensus.cc:374] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:02.249414 16809 raft_consensus.cc:729] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0428b465bb6c423d8c39ac4a8191e5c1, State: Initialized, Role: FOLLOWER
I20260619 12:08:02.249462 16809 consensus_queue.cc:260] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.249554 16809 ts_tablet_manager.cc:1428] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:02.250948 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.246063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34723 (local address 127.0.0.1:53822)
0619 12:08:02.246131 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:02.246145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:02.246188 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:02.246827 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:02.246828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:02.246832 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:02.246910 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.246912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.248043 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.248044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.249312 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.249317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.250538 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.250540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.250668 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.250675 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:08:02.250754 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:08:02.250828 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":10}
I20260619 12:08:02.250950 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.246137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44917 (local address 127.0.0.1:53524)
0619 12:08:02.246404 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:02.246415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:02.246474 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:02.246925 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:02.246927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:02.246929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:02.247003 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.247005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.248349 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.248350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.249609 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.249611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.250538 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.250540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.250676 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.250679 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:02.250754 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:08:02.250828 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":81,"thread_start_us":100,"threads_started":1}
I20260619 12:08:02.251004 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.246478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.0.203.131:44917)
0619 12:08:02.246715 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.246721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.246732 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:02.246775 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.246778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.246808 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.247071 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:02.247167 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.248247 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.249721 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.250419 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.250494 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.250571 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.250818 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.250886 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":105,"threads_started":1}
I20260619 12:08:02.251004 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.246380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.0.203.130:34723)
0619 12:08:02.246640 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.246646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.246659 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:08:02.246706 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.246708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.246724 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.246817 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:02.246979 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.247960 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.249556 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.250403 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.250494 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.250572 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.250818 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.250886 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":123,"threads_started":1}
I20260619 12:08:02.251993 17121 tablet_service.cc:1467] Processing CreateTablet for tablet fa8c98e18d614328a35e325ac025f9b0 (DEFAULT_TABLE table=test-table [id=339f067a1b6c43aa9bc2bb2f5ae098f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:02.252103 16989 tablet_service.cc:1467] Processing CreateTablet for tablet fa8c98e18d614328a35e325ac025f9b0 (DEFAULT_TABLE table=test-table [id=339f067a1b6c43aa9bc2bb2f5ae098f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:02.252504 16989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa8c98e18d614328a35e325ac025f9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:02.252506 17121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa8c98e18d614328a35e325ac025f9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:02.255764 17197 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Bootstrap starting.
I20260619 12:08:02.256068 17198 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Bootstrap starting.
I20260619 12:08:02.256690 17197 tablet_bootstrap.cc:654] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:02.256803 17198 tablet_bootstrap.cc:654] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:02.257309 17197 log.cc:824] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:02.257368 17198 log.cc:824] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:02.258262 17197 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: No bootstrap required, opened a new log
I20260619 12:08:02.258339 17197 ts_tablet_manager.cc:1397] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:08:02.258340 17198 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: No bootstrap required, opened a new log
I20260619 12:08:02.258417 17198 ts_tablet_manager.cc:1397] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:02.260126 17197 raft_consensus.cc:348] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.260313 17197 raft_consensus.cc:374] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:02.260350 17197 raft_consensus.cc:729] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa7a0029147f4adea28b64769d6fd208, State: Initialized, Role: FOLLOWER
I20260619 12:08:02.260226 17198 raft_consensus.cc:348] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.260401 17198 raft_consensus.cc:374] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:02.260455 17198 raft_consensus.cc:729] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d30fe60abe2d448bbaaa3cbcd1d06033, State: Initialized, Role: FOLLOWER
I20260619 12:08:02.260556 17197 consensus_queue.cc:260] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.260605 17198 consensus_queue.cc:260] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.260895 17197 ts_tablet_manager.cc:1428] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:08:02.260957 17055 heartbeater.cc:503] Master 127.0.203.190:45807 was elected leader, sending a full tablet report...
I20260619 12:08:02.260978 17187 heartbeater.cc:503] Master 127.0.203.190:45807 was elected leader, sending a full tablet report...
I20260619 12:08:02.261225 17198 ts_tablet_manager.cc:1428] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260619 12:08:02.306723 17056 tablet.cc:2369] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:02.389014 16923 raft_consensus.cc:482] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:02.389165 16923 raft_consensus.cc:504] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.389521 16923 leader_election.cc:288] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723), d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
I20260619 12:08:02.393209 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.389849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:48007 (local address 127.0.203.130:34723)
0619 12:08:02.389941 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.389944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.389955 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:02.389986 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.389988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.390000 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.390068 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:02.390308 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.390871 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.392072 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.392677 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.392865 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.392885 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.393093 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.393136 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:08:02.393249 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.389744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34723 (local address 127.0.203.129:48007)
0619 12:08:02.389787 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:02.389801 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:02.389862 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:02.390123 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:02.390125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:02.390128 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:02.390223 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.390226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.390967 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.390968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.391947 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.391950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.392875 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.392877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.392976 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.392983 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:08:02.393054 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:08:02.393118 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":12}
I20260619 12:08:02.393805 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.389841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44917 (local address 127.0.203.129:34321)
0619 12:08:02.390041 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:02.390053 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:02.390126 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:02.390266 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:02.390266 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:02.390268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:02.390639 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.390641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.391406 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.391407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.392663 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:02.392668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.393538 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.393539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:02.393603 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.393609 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:08:02.393665 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:08:02.393715 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":132,"threads_started":1}
I20260619 12:08:02.393833 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.390070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:34321 (local address 127.0.203.131:44917)
0619 12:08:02.390127 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.390128 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.390165 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:08:02.390190 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.390191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.390198 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.390248 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:02.390731 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.391260 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.392874 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.393457 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.393537 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.393549 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.393683 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.393731 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:08:02.393795 17009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8c98e18d614328a35e325ac025f9b0" candidate_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa7a0029147f4adea28b64769d6fd208" is_pre_election: true
I20260619 12:08:02.393975 17009 raft_consensus.cc:2454] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0428b465bb6c423d8c39ac4a8191e5c1 in term 0.
I20260619 12:08:02.394202 17141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8c98e18d614328a35e325ac025f9b0" candidate_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" is_pre_election: true
I20260619 12:08:02.394240 16805 leader_election.cc:302] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1, fa7a0029147f4adea28b64769d6fd208; no voters:
I20260619 12:08:02.394447 17141 raft_consensus.cc:2454] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0428b465bb6c423d8c39ac4a8191e5c1 in term 0.
I20260619 12:08:02.394503 16923 raft_consensus.cc:2781] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:08:02.394634 16923 raft_consensus.cc:482] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:08:02.394665 16923 raft_consensus.cc:3037] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:02.395576 16923 raft_consensus.cc:504] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.395758 16923 leader_election.cc:288] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 1 election: Requested vote from peers fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723), d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
I20260619 12:08:02.395994 17141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8c98e18d614328a35e325ac025f9b0" candidate_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033"
W20260619 12:08:02.396052 16918 tablet.cc:2369] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:02.396100 17141 raft_consensus.cc:3037] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:02.396375 17009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8c98e18d614328a35e325ac025f9b0" candidate_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa7a0029147f4adea28b64769d6fd208"
I20260619 12:08:02.396453 17009 raft_consensus.cc:3037] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:02.397318 17141 raft_consensus.cc:2454] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0428b465bb6c423d8c39ac4a8191e5c1 in term 1.
I20260619 12:08:02.397423 17009 raft_consensus.cc:2454] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0428b465bb6c423d8c39ac4a8191e5c1 in term 1.
I20260619 12:08:02.397622 16803 leader_election.cc:302] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1, d30fe60abe2d448bbaaa3cbcd1d06033; no voters:
I20260619 12:08:02.397840 16923 raft_consensus.cc:2781] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:02.398118 16923 raft_consensus.cc:686] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 LEADER]: Becoming Leader. State: Replica: 0428b465bb6c423d8c39ac4a8191e5c1, State: Running, Role: LEADER
I20260619 12:08:02.398231 16923 consensus_queue.cc:237] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:02.399147 16729 catalog_manager.cc:5495] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129). New cstate: current_term: 1 leader_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } health_report { overall_health: HEALTHY } } }
I20260619 12:08:02.437777 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.435161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.0.203.129:41991)
0619 12:08:02.435352 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:02.435354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:02.435387 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:08:02.435441 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:02.435442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:02.435470 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:02.435549 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:02.435696 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.436205 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.436953 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:02.437165 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:02.437222 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:02.437397 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:02.437523 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:08:02.437592 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:02.437647 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 12:08:02.437678 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260619 12:08:02.439878 17141 raft_consensus.cc:1261] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Refusing update from remote peer 0428b465bb6c423d8c39ac4a8191e5c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:02.439877 17009 raft_consensus.cc:1261] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 1 FOLLOWER]: Refusing update from remote peer 0428b465bb6c423d8c39ac4a8191e5c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:02.440291 16923 consensus_queue.cc:1035] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:02.440487 17204 consensus_queue.cc:1035] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:02.443660 17208 mvcc.cc:205] Tried to move back new op lower bound from 7298543134470103040 to 7298543134304038912. Current Snapshot: MvccSnapshot[applied={T|T < 7298543134470103040}]
I20260619 12:08:02.444177 17209 mvcc.cc:205] Tried to move back new op lower bound from 7298543134470103040 to 7298543134304038912. Current Snapshot: MvccSnapshot[applied={T|T < 7298543134470103040}]
I20260619 12:08:02.443738 17210 mvcc.cc:205] Tried to move back new op lower bound from 7298543134470103040 to 7298543134304038912. Current Snapshot: MvccSnapshot[applied={T|T < 7298543134470103040}]
I20260619 12:08:02.447285 16728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:02.447459 16728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:02.448521 16728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Sending DeleteTablet for 3 replicas of tablet fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:02.448945 16851 tablet_service.cc:1514] Processing DeleteTablet for tablet fa8c98e18d614328a35e325ac025f9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:02 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:42192
I20260619 12:08:02.448951 17121 tablet_service.cc:1514] Processing DeleteTablet for tablet fa8c98e18d614328a35e325ac025f9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:02 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:53524
I20260619 12:08:02.448982 16989 tablet_service.cc:1514] Processing DeleteTablet for tablet fa8c98e18d614328a35e325ac025f9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:02 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:53822
I20260619 12:08:02.449286 17219 tablet_replica.cc:330] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: stopping tablet replica
I20260619 12:08:02.449370 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 16767
I20260619 12:08:02.449426 17219 raft_consensus.cc:2229] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:02.449564 17219 raft_consensus.cc:2258] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:02.450788 17219 ts_tablet_manager.cc:1905] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:02.451133 17217 tablet_replica.cc:330] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: stopping tablet replica
I20260619 12:08:02.451290 17217 raft_consensus.cc:2229] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:02.451454 17217 raft_consensus.cc:2258] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:02.452322 17217 ts_tablet_manager.cc:1905] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:02.452791 17219 ts_tablet_manager.cc:1918] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:02.452872 17219 log.cc:1196] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:02.453141 17219 ts_tablet_manager.cc:1939] T fa8c98e18d614328a35e325ac025f9b0 P fa7a0029147f4adea28b64769d6fd208: Deleting consensus metadata
I20260619 12:08:02.453660 16715 catalog_manager.cc:4841] TS fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723): tablet fa8c98e18d614328a35e325ac025f9b0 (table test-table [id=339f067a1b6c43aa9bc2bb2f5ae098f9]) successfully deleted
I20260619 12:08:02.454113 17217 ts_tablet_manager.cc:1918] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:02.454200 17217 log.cc:1196] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:02.454491 17217 ts_tablet_manager.cc:1939] T fa8c98e18d614328a35e325ac025f9b0 P d30fe60abe2d448bbaaa3cbcd1d06033: Deleting consensus metadata
I20260619 12:08:02.455019 16713 catalog_manager.cc:4841] TS d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917): tablet fa8c98e18d614328a35e325ac025f9b0 (table test-table [id=339f067a1b6c43aa9bc2bb2f5ae098f9]) successfully deleted
W20260619 12:08:02.455837 16714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:41991 (error 108)
I20260619 12:08:02.455992 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 16919
I20260619 12:08:02.456943 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:02.456794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:41991 (local address 127.0.0.1:42202)
0619 12:08:02.456861 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:02.456902 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:41991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 12:08:02.457082 16714 catalog_manager.cc:4584] TS 0428b465bb6c423d8c39ac4a8191e5c1 (127.0.203.129:41991): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa8c98e18d614328a35e325ac025f9b0: Network error: Client connection negotiation failed: client connection to 127.0.203.129:41991: connect: Connection refused (error 111)
I20260619 12:08:02.460983 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 17058
I20260619 12:08:02.466986 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 16696
I20260619 12:08:22.473750 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:45807
--webserver_interface=127.0.203.190
--webserver_port=39079
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:22.562376 17221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:22.562647 17221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:22.562686 17221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:22.564086 17221 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:08:22.564149 17221 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:08:22.564167 17221 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:08:22.564182 17221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:22.564199 17221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:22.564216 17221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:22.564231 17221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:22.564248 17221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:22.566644 17221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:45807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=39079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:22.567132 17221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:22.567471 17221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:22.575044 17227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:22.575080 17221 server_base.cc:1034] running on GCE node
W20260619 12:08:22.575034 17226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:22.575034 17229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:22.575770 17221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:22.576105 17221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:22.577387 17221 hybrid_clock.cc:648] HybridClock initialized: now 1781870902577280 us; error 67 us; skew 500 ppm
Jun 19 12:08:22 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:22.580546 17221 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:08:22.581115 17221 webserver.cc:460] Webserver started at http://127.0.203.190:39079/ using document root <none> and password file <none>
I20260619 12:08:22.581341 17221 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:22.581391 17221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:22.583017 17221 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:22.583886 17236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:22.584168 17221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:08:22.584285 17221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "f607b569dbc0410084a04bab0782ed86"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "94258fbf97a4beb238d8229450454018"
server_key_iv: "5ecfeb30fd914a24e3731e83fa7f79c5"
server_key_version: "encryptionkey@0"
I20260619 12:08:22.584719 17221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:22.597949 17221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:22.600440 17221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:22.600590 17221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:22.605813 17288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:45807 every 8 connection(s)
I20260619 12:08:22.605901 17221 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:45807
I20260619 12:08:22.606511 17221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:22.608479 17289 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:08:22.609318 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Bootstrap starting.
I20260619 12:08:22.610148 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 17221
I20260619 12:08:22.610487 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:41991
--local_ip_for_outbound_sockets=127.0.203.129
--tserver_master_addrs=127.0.203.190:45807
--webserver_port=37343
--webserver_interface=127.0.203.129
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:08:22.620652 17289 log.cc:824] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:22.623582 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:08:22.623941 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Bootstrap complete.
I20260619 12:08:22.626390 17289 raft_consensus.cc:348] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:22.626685 17289 raft_consensus.cc:729] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f607b569dbc0410084a04bab0782ed86, State: Initialized, Role: FOLLOWER
I20260619 12:08:22.626886 17289 consensus_queue.cc:260] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:22.626976 17289 raft_consensus.cc:388] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:22.627060 17289 raft_consensus.cc:482] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:22.627115 17289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:08:22.628520 17289 raft_consensus.cc:504] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:22.628643 17289 leader_election.cc:302] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f607b569dbc0410084a04bab0782ed86; no voters:
I20260619 12:08:22.628849 17289 leader_election.cc:288] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 12:08:22.628907 17293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:08:22.629053 17293 raft_consensus.cc:686] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [term 3 LEADER]: Becoming Leader. State: Replica: f607b569dbc0410084a04bab0782ed86, State: Running, Role: LEADER
I20260619 12:08:22.629151 17293 consensus_queue.cc:237] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } }
I20260619 12:08:22.629192 17289 sys_catalog.cc:564] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:22.629531 17293 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f607b569dbc0410084a04bab0782ed86. Latest consensus state: current_term: 3 leader_uuid: "f607b569dbc0410084a04bab0782ed86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } } }
I20260619 12:08:22.629611 17293 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:22.629597 17294 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f607b569dbc0410084a04bab0782ed86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607b569dbc0410084a04bab0782ed86" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 45807 } } }
I20260619 12:08:22.629792 17294 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:22.629967 17298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:22.631255 17298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:22.631420 17298 catalog_manager.cc:1259] Loaded cluster ID: 5d54e1d7263f4629a9e46af626a4c5cd
I20260619 12:08:22.631461 17298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:22.632874 17298 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:22.633247 17298 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Loaded TSK: 0
I20260619 12:08:22.633311 17298 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f607b569dbc0410084a04bab0782ed86: Loaded TSK: 1
F20260619 12:08:22.633587 17298 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe7b80e59ad google::LogMessage::Fail() at ??:0
@ 0x7fe7b80e888a google::LogMessage::SendToLog() at ??:0
@ 0x7fe7b80e54de google::LogMessage::Flush() at ??:0
@ 0x7fe7b80e6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe7ba852104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe7ba84fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe7c00431cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe7c004348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe7c0043594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe7c0009c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe7c0013ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe7b9fb30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe7b9fa8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe7bae0c6db start_thread at ??:0
@ 0x7fe7b6be361f clone at ??:0
W20260619 12:08:22.729874 17291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:22.730196 17291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:22.730237 17291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:22.732748 17291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:22.732861 17291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:22.732913 17291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:22.735733 17291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:41991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=37343
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:22.736093 17291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:22.736397 17291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:22.742653 17333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:22.742728 17330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:22.742915 17331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:22.742852 17291 server_base.cc:1034] running on GCE node
I20260619 12:08:22.743247 17291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:22.743712 17291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:22.745069 17291 hybrid_clock.cc:648] HybridClock initialized: now 1781870902745008 us; error 72 us; skew 500 ppm
Jun 19 12:08:22 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:22.749143 17291 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:08:22.749933 17291 webserver.cc:460] Webserver started at http://127.0.203.129:37343/ using document root <none> and password file <none>
I20260619 12:08:22.750236 17291 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:22.750321 17291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:22.752699 17291 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:22.753850 17340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:22.754156 17291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:22.754298 17291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "0428b465bb6c423d8c39ac4a8191e5c1"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "26d7c80f85ee53af2ef36699a0433d63"
server_key_iv: "7421fe71eaa5314deee3a143ad8b2006"
server_key_version: "encryptionkey@0"
I20260619 12:08:22.754843 17291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:22.781558 17291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:22.784888 17291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:22.785061 17291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:22.785373 17291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:22.786160 17347 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:08:22.788250 17291 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:08:22.788365 17291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 12:08:22.788409 17291 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:08:22.789844 17291 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:08:22.789919 17291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 12:08:22.790025 17347 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap starting.
I20260619 12:08:22.798930 17291 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:41991
I20260619 12:08:22.799422 17454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:41991 every 8 connection(s)
I20260619 12:08:22.799535 17291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:08:22.803342 17347 log.cc:824] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:22.806449 17347 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:08:22.806799 17347 tablet_bootstrap.cc:492] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Bootstrap complete.
I20260619 12:08:22.807753 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 17291
I20260619 12:08:22.807785 17347 ts_tablet_manager.cc:1397] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.010s
I20260619 12:08:22.808120 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:34723
--local_ip_for_outbound_sockets=127.0.203.130
--tserver_master_addrs=127.0.203.190:45807
--webserver_port=46575
--webserver_interface=127.0.203.130
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:08:22.809660 17347 raft_consensus.cc:348] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:22.810058 17347 raft_consensus.cc:729] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0428b465bb6c423d8c39ac4a8191e5c1, State: Initialized, Role: FOLLOWER
I20260619 12:08:22.810283 17347 consensus_queue.cc:260] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:22.810655 17347 ts_tablet_manager.cc:1428] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
W20260619 12:08:22.919202 17457 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:08:22.800687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:60867)
0619 12:08:22.801034 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:22.801105 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:22.801834 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:22.918908 (+117074us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":180,"threads_started":1}
W20260619 12:08:22.919596 17455 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:45807 (0 consecutive failures): Network error: Failed to ping master at 127.0.203.190:45807: Client connection negotiation failed: client connection to 127.0.203.190:45807: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:08:22.920136 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:22.920007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:51399)
0619 12:08:22.920063 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:22.920109 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 12:08:22.920701 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:22.920603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:45047)
0619 12:08:22.920657 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:22.920683 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:08:22.920862 17455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:08:22.931202 17459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:22.931512 17459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:22.931548 17459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:22.932842 17459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:22.932888 17459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:22.932924 17459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:22.934535 17459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:34723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=46575
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:22.934762 17459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:22.934969 17459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:22.940862 17466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:22.940853 17465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:22.940853 17468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:22.940897 17459 server_base.cc:1034] running on GCE node
I20260619 12:08:22.941366 17459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:22.941618 17459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:22.942812 17459 hybrid_clock.cc:648] HybridClock initialized: now 1781870902942784 us; error 45 us; skew 500 ppm
Jun 19 12:08:22 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:22.945663 17459 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:08:22.946218 17459 webserver.cc:460] Webserver started at http://127.0.203.130:46575/ using document root <none> and password file <none>
I20260619 12:08:22.946472 17459 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:22.946529 17459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:22.948272 17459 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:08:22.949124 17475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:22.949339 17459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:22.949440 17459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "fa7a0029147f4adea28b64769d6fd208"
format_stamp: "Formatted at 2026-06-19 12:07:40 on dist-test-slave-44vj"
server_key: "aa88339d398208a029c99fdfa77ca97e"
server_key_iv: "5b96641eebeb1cb0ead84bc7281db574"
server_key_version: "encryptionkey@0"
I20260619 12:08:22.949911 17459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:22.968861 17459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:22.971274 17459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:22.971540 17459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:22.971801 17459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:22.972157 17459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:22.972199 17459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:22.972231 17459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:22.972249 17459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:22.979166 17459 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:34723
I20260619 12:08:22.979305 17588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:34723 every 8 connection(s)
I20260619 12:08:22.979681 17459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:22.981159 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:22.980589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:34141)
0619 12:08:22.980800 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:22.980909 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
W20260619 12:08:22.981714 17589 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:45807 (0 consecutive failures): Network error: Failed to ping master at 127.0.203.190:45807: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
I20260619 12:08:22.982124 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:22.981993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:49255)
0619 12:08:22.982061 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:22.982101 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 12:08:22.982434 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:22.982318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:58353)
0619 12:08:22.982383 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:22.982412 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 12:08:22.982556 17589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:08:22.987094 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 17459
I20260619 12:08:22.987433 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:44917
--local_ip_for_outbound_sockets=127.0.203.131
--tserver_master_addrs=127.0.203.190:45807
--webserver_port=33009
--webserver_interface=127.0.203.131
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:23.076833 17592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:23.077157 17592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:23.077313 17592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:23.079432 17592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:23.079505 17592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:23.079556 17592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:23.081537 17592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:44917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=33009
--tserver_master_addrs=127.0.203.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:23.081827 17592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:23.082087 17592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:23.087608 17597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:23.087783 17592 server_base.cc:1034] running on GCE node
W20260619 12:08:23.087641 17600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:23.087741 17598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:23.088299 17592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:23.088645 17592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:23.089804 17592 hybrid_clock.cc:648] HybridClock initialized: now 1781870903089792 us; error 33 us; skew 500 ppm
Jun 19 12:08:23 dist-test-slave-44vj krb5kdc[16145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870903, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:23.092579 17592 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:08:23.093128 17592 webserver.cc:460] Webserver started at http://127.0.203.131:33009/ using document root <none> and password file <none>
I20260619 12:08:23.093367 17592 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:23.093420 17592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:23.094983 17592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:23.095752 17607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:23.096004 17592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:08:23.096119 17592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "d30fe60abe2d448bbaaa3cbcd1d06033"
format_stamp: "Formatted at 2026-06-19 12:07:41 on dist-test-slave-44vj"
server_key: "614e0ecb1d27b7da277a8ae28eed0aa1"
server_key_iv: "3dbd2939e2f1daec9096f3ea486ad80e"
server_key_version: "encryptionkey@0"
I20260619 12:08:23.096506 17592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:23.108243 17592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:23.110888 17592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:23.111037 17592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:23.111267 17592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:23.111665 17592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:23.111711 17592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:23.111734 17592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:23.111745 17592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:23.118973 17592 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:44917
I20260619 12:08:23.119124 17720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:44917 every 8 connection(s)
I20260619 12:08:23.119514 17592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:08:23.120951 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.120370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:60891)
0619 12:08:23.120591 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:23.120726 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":96,"threads_started":1}
W20260619 12:08:23.121441 17721 heartbeater.cc:644] Failed to heartbeat to 127.0.203.190:45807 (0 consecutive failures): Network error: Failed to ping master at 127.0.203.190:45807: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
I20260619 12:08:23.121791 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.121683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:50537)
0619 12:08:23.121736 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:23.121770 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 12:08:23.122035 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.121975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:43239)
0619 12:08:23.122004 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:23.122022 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260619 12:08:23.122188 17721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:08:23.124156 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 17592
I20260619 12:08:23.921758 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.921388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:41251)
0619 12:08:23.921603 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:23.921673 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":76,"threads_started":1}
I20260619 12:08:23.954550 17726 raft_consensus.cc:482] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:23.954674 17726 raft_consensus.cc:504] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:23.954993 17726 leader_election.cc:288] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723), d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
Jun 19 12:08:23 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.131@KRBTEST.COM
Jun 19 12:08:23 dist-test-slave-44vj krb5kdc[16145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.130@KRBTEST.COM
I20260619 12:08:23.972496 17727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.955316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:45891 (local address 127.0.203.130:34723)
0619 12:08:23.955550 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:23.955568 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:23.955597 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:08:23.956257 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:23.956260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:23.956329 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:23.956499 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:23.957361 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.958437 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.959540 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.960012 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.960136 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:23.968550 (+ 8414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:23.968558 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:23.968558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:23.968585 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:23.970489 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:23.971043 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:23.971047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:23.971048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:23.971174 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:23.971489 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:23.971490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:23.971490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:23.971814 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:23.971919 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:23.972199 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 12:08:23.972368 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":98,"threads_started":1}
I20260619 12:08:23.972501 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.955415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:54883 (local address 127.0.203.131:44917)
0619 12:08:23.955585 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:23.955596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:23.955623 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:08:23.956166 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:23.956168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:23.956226 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:23.956360 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:23.957392 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.958321 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.959567 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.960001 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.960105 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:23.968117 (+ 8012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:23.968125 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:23.968126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:23.968150 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:23.970139 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:23.970789 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:23.970794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:23.970794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:23.970940 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:23.971402 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:23.971403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:23.971403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:23.971735 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:23.971806 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:23.972199 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 12:08:23.972364 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260619 12:08:23.972496 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.955280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44917 (local address 127.0.203.129:54883)
0619 12:08:23.955488 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:23.955501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:23.955571 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:23.956449 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:23.956453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:23.956950 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:23.957222 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:23.957225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.958446 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.958447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:23.959445 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:23.959449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.960102 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.960103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:23.960232 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:23.965728 (+ 5496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:23.965748 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:23.967819 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:23.970268 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:23.970283 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:23.970286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:23.970642 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:23.971032 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:23.971033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:23.971033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:23.971248 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:23.971815 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:23.971824 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:23.971988 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 12:08:23.972090 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 12:08:23.972380 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":1,"thread_start_us":129,"threads_started":1}
I20260619 12:08:23.972949 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.955182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:34723 (local address 127.0.203.129:45891)
0619 12:08:23.955232 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:23.955245 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:23.955316 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:23.956498 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:23.956500 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:23.956958 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:23.957180 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:23.957187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.958559 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.958561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:23.959443 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:23.959445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:23.960103 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:23.960104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:23.960232 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:23.965723 (+ 5491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:23.965736 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:23.968325 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:23.970642 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:23.970644 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:23.970645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:23.970938 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:23.971286 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:23.971288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:23.971289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:23.971409 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:23.971910 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:23.971911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:23.971992 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 12:08:23.972092 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 12:08:23.972486 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":276}
W20260619 12:08:23.973742 17341 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
W20260619 12:08:23.974030 17343 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:23.974097 17343 leader_election.cc:302] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1; no voters: d30fe60abe2d448bbaaa3cbcd1d06033, fa7a0029147f4adea28b64769d6fd208
I20260619 12:08:23.974221 17726 raft_consensus.cc:2726] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:08:23.983502 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:23.983084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:34067)
0619 12:08:23.983328 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:23.983451 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":127,"threads_started":1}
I20260619 12:08:24.123087 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:24.122714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:41917)
0619 12:08:24.122949 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:24.123033 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
I20260619 12:08:24.922832 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:24.922523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:57429)
0619 12:08:24.922695 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:24.922766 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260619 12:08:24.984454 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:24.984137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:33517)
0619 12:08:24.984338 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:24.984399 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
I20260619 12:08:25.124308 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:25.123983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:50873)
0619 12:08:25.124175 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:25.124242 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260619 12:08:25.828106 17735 raft_consensus.cc:482] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:25.828208 17735 raft_consensus.cc:504] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:25.828408 17735 leader_election.cc:288] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723), d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
W20260619 12:08:25.828977 17341 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
W20260619 12:08:25.829095 17343 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:25.829180 17343 leader_election.cc:302] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1; no voters: d30fe60abe2d448bbaaa3cbcd1d06033, fa7a0029147f4adea28b64769d6fd208
I20260619 12:08:25.829270 17735 raft_consensus.cc:2726] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:08:25.923866 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:25.923553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:40557)
0619 12:08:25.923751 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:25.923814 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
I20260619 12:08:25.985437 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:25.985133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:54221)
0619 12:08:25.985316 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:25.985378 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
I20260619 12:08:26.125180 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:26.124894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:34649)
0619 12:08:26.125071 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:26.125117 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260619 12:08:26.924754 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:26.924454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:35241)
0619 12:08:26.924617 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:26.924686 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260619 12:08:26.986546 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:26.986180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:44771)
0619 12:08:26.986418 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:26.986489 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
I20260619 12:08:27.126278 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:27.125881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:44499)
0619 12:08:27.126134 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:27.126197 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260619 12:08:27.925894 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:27.925546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:59327)
0619 12:08:27.925767 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:27.925841 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":99,"threads_started":1}
I20260619 12:08:27.987637 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:27.987243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:59529)
0619 12:08:27.987484 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:27.987552 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
I20260619 12:08:28.127348 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:28.126957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:35681)
0619 12:08:28.127204 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:28.127281 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":82,"threads_started":1}
I20260619 12:08:28.927033 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:28.926711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:60395)
0619 12:08:28.926916 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:28.926964 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":107,"threads_started":1}
I20260619 12:08:28.988771 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:28.988412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:47247)
0619 12:08:28.988652 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:28.988718 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
I20260619 12:08:28.997673 17747 raft_consensus.cc:482] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:28.997818 17747 raft_consensus.cc:504] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:28.998044 17747 leader_election.cc:288] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723), d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
W20260619 12:08:28.998616 17341 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
W20260619 12:08:28.998765 17343 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:28.998834 17343 leader_election.cc:302] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1; no voters: d30fe60abe2d448bbaaa3cbcd1d06033, fa7a0029147f4adea28b64769d6fd208
I20260619 12:08:28.998919 17747 raft_consensus.cc:2726] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:08:29.128489 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:29.128146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:44931)
0619 12:08:29.128335 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:29.128419 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
I20260619 12:08:29.928181 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:29.927878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:40907)
0619 12:08:29.928051 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:29.928095 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260619 12:08:29.989810 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:29.989480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:43767)
0619 12:08:29.989692 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:29.989757 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
I20260619 12:08:30.129596 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:30.129220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:44525)
0619 12:08:30.129457 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:30.129531 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
I20260619 12:08:30.590168 17752 raft_consensus.cc:482] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:30.590286 17752 raft_consensus.cc:504] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:30.590463 17752 leader_election.cc:288] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723), d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
W20260619 12:08:30.590957 17343 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
W20260619 12:08:30.591045 17341 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:30.591126 17341 leader_election.cc:302] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1; no voters: d30fe60abe2d448bbaaa3cbcd1d06033, fa7a0029147f4adea28b64769d6fd208
I20260619 12:08:30.591235 17752 raft_consensus.cc:2726] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:08:30.929131 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:30.928785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:47033)
0619 12:08:30.929009 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:30.929076 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":90,"threads_started":1}
I20260619 12:08:30.990626 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:30.990328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:45781)
0619 12:08:30.990500 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:30.990560 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260619 12:08:31.130625 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:31.130255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:58815)
0619 12:08:31.130507 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:31.130555 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":171,"threads_started":1}
I20260619 12:08:31.930142 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:31.929824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:57793)
0619 12:08:31.930008 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:31.930077 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":78,"threads_started":1}
I20260619 12:08:31.991573 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:31.991246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:38421)
0619 12:08:31.991441 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:31.991508 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260619 12:08:32.131753 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:32.131452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:47329)
0619 12:08:32.131636 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:32.131691 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260619 12:08:32.931334 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:32.930867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:33055)
0619 12:08:32.931210 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:32.931268 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":115,"threads_started":1}
I20260619 12:08:32.992501 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:32.992222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:53419)
0619 12:08:32.992380 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:32.992438 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260619 12:08:33.132725 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:33.132367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:34277)
0619 12:08:33.132566 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:33.132641 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
I20260619 12:08:33.932332 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:33.932008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:56623)
0619 12:08:33.932214 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:33.932269 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260619 12:08:33.993551 17763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:33.993247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:51509)
0619 12:08:33.993440 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:33.993485 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":99,"threads_started":1}
I20260619 12:08:34.133783 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:34.133417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:36843)
0619 12:08:34.133634 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:34.133708 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":102,"threads_started":1}
I20260619 12:08:34.933228 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:34.932953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:52497)
0619 12:08:34.933109 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:34.933164 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260619 12:08:34.994521 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:34.994185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:53843)
0619 12:08:34.994399 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:34.994466 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":80,"threads_started":1}
I20260619 12:08:35.134694 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:35.134385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:54859)
0619 12:08:35.134544 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:35.134632 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260619 12:08:35.414260 17768 raft_consensus.cc:482] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:35.414392 17768 raft_consensus.cc:504] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa7a0029147f4adea28b64769d6fd208" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 34723 } } peers { permanent_uuid: "d30fe60abe2d448bbaaa3cbcd1d06033" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44917 } } peers { permanent_uuid: "0428b465bb6c423d8c39ac4a8191e5c1" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 41991 } }
I20260619 12:08:35.414597 17768 leader_election.cc:288] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723), d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917)
W20260619 12:08:35.415071 17343 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa7a0029147f4adea28b64769d6fd208 (127.0.203.130:34723): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
W20260619 12:08:35.415194 17341 leader_election.cc:341] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d30fe60abe2d448bbaaa3cbcd1d06033 (127.0.203.131:44917): Not found: Tablet not found: fa8c98e18d614328a35e325ac025f9b0
I20260619 12:08:35.415254 17341 leader_election.cc:302] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0428b465bb6c423d8c39ac4a8191e5c1; no voters: d30fe60abe2d448bbaaa3cbcd1d06033, fa7a0029147f4adea28b64769d6fd208
I20260619 12:08:35.415316 17768 raft_consensus.cc:2726] T fa8c98e18d614328a35e325ac025f9b0 P 0428b465bb6c423d8c39ac4a8191e5c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:08:35.934196 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:35.933851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:40961)
0619 12:08:35.934082 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:35.934148 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260619 12:08:35.995509 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:35.995128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:45111)
0619 12:08:35.995321 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:35.995430 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
I20260619 12:08:36.135643 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:36.135295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:46729)
0619 12:08:36.135531 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:36.135596 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":145,"threads_started":1}
I20260619 12:08:36.935163 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:36.934860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:50141)
0619 12:08:36.935049 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:36.935114 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260619 12:08:36.996443 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:36.996125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:33887)
0619 12:08:36.996306 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:36.996377 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
I20260619 12:08:37.136520 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:37.136202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:57075)
0619 12:08:37.136384 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:37.136456 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260619 12:08:37.936088 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:37.935845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.129:49139)
0619 12:08:37.935993 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:37.936044 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260619 12:08:37.997440 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:37.997135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.130:38031)
0619 12:08:37.997314 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:37.997367 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
I20260619 12:08:38.131920 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 17291
I20260619 12:08:38.136756 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 17459
I20260619 12:08:38.138322 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:38.137529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:45807 (local address 127.0.203.131:46431)
0619 12:08:38.138236 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:38.138271 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.190:45807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":67,"threads_started":1}
I20260619 12:08:38.141721 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 17592
2026-06-19T12:08:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58829 ms)
[----------] 30 tests from SecurityITest (244523 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:38 dist-test-slave-44vj krb5kdc[17780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:38 dist-test-slave-44vj krb5kdc[17780](info): set up 2 sockets
Jun 19 12:08:38 dist-test-slave-44vj krb5kdc[17780](info): commencing operation
krb5kdc: starting...
W20260619 12:08:39.123247 814 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:08:39.255146 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:35915
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:35915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:39.344763 17793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:39.345034 17793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:39.345069 17793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:39.346338 17793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:39.346385 17793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:39.346403 17793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:39.346418 17793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:39.346434 17793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:39.348073 17793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:35915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:35915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:39.348345 17793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:39.348579 17793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:39.354349 17798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:39.354527 17793 server_base.cc:1034] running on GCE node
W20260619 12:08:39.354375 17799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:39.354413 17801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:39.355073 17793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:39.355446 17793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:39.356624 17793 hybrid_clock.cc:648] HybridClock initialized: now 1781870919356597 us; error 52 us; skew 500 ppm
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:39.359650 17793 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:08:39.360152 17793 webserver.cc:460] Webserver started at http://127.0.203.190:33381/ using document root <none> and password file <none>
I20260619 12:08:39.360388 17793 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:39.360440 17793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:39.360582 17793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:39.361737 17793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "d3456f177f374c94a3f38804e8e97f15"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "e5b398e9519101a7e25cbea791ce6e15"
server_key_iv: "82293415600934aafef39c077d1e1d64"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.362110 17793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "d3456f177f374c94a3f38804e8e97f15"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "e5b398e9519101a7e25cbea791ce6e15"
server_key_iv: "82293415600934aafef39c077d1e1d64"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.363708 17793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:39.364817 17808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.365056 17793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:39.365171 17793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "d3456f177f374c94a3f38804e8e97f15"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "e5b398e9519101a7e25cbea791ce6e15"
server_key_iv: "82293415600934aafef39c077d1e1d64"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.365269 17793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:39.380868 17793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:39.383351 17793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:39.383538 17793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:39.388422 17793 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:35915
I20260619 12:08:39.388484 17860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:35915 every 8 connection(s)
I20260619 12:08:39.388880 17793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:39.389899 17861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:39.391220 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 17793
I20260619 12:08:39.391325 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:39.391592 814 external_mini_cluster.cc:1428] Setting key cf99b2c37bbb2b8dc876948dbbe4443f
I20260619 12:08:39.392738 17861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15: Bootstrap starting.
I20260619 12:08:39.393491 17861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:39.393800 17861 log.cc:824] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:39.394528 17861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15: No bootstrap required, opened a new log
I20260619 12:08:39.396812 17861 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3456f177f374c94a3f38804e8e97f15" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35915 } }
I20260619 12:08:39.397029 17861 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:39.397064 17861 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3456f177f374c94a3f38804e8e97f15, State: Initialized, Role: FOLLOWER
I20260619 12:08:39.397225 17861 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3456f177f374c94a3f38804e8e97f15" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35915 } }
I20260619 12:08:39.397317 17861 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:39.397351 17861 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:39.397390 17861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:39.398046 17861 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3456f177f374c94a3f38804e8e97f15" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35915 } }
I20260619 12:08:39.398136 17861 leader_election.cc:302] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3456f177f374c94a3f38804e8e97f15; no voters:
I20260619 12:08:39.398324 17861 leader_election.cc:288] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:39.398423 17866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:39.398592 17861 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:39.398758 17866 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [term 1 LEADER]: Becoming Leader. State: Replica: d3456f177f374c94a3f38804e8e97f15, State: Running, Role: LEADER
I20260619 12:08:39.398969 17866 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3456f177f374c94a3f38804e8e97f15" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35915 } }
I20260619 12:08:39.399610 17868 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3456f177f374c94a3f38804e8e97f15. Latest consensus state: current_term: 1 leader_uuid: "d3456f177f374c94a3f38804e8e97f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3456f177f374c94a3f38804e8e97f15" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35915 } } }
I20260619 12:08:39.399729 17868 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:39.399802 17867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3456f177f374c94a3f38804e8e97f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3456f177f374c94a3f38804e8e97f15" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35915 } } }
I20260619 12:08:39.399883 17867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15 [sys.catalog]: This master's current role is: LEADER
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:39.400869 17879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:39.401401 17879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:39.403431 17879 catalog_manager.cc:1347] Generated new cluster ID: 5f48a555f01b4f70891ef51265983182
I20260619 12:08:39.403491 17879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:39.404315 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:39.392303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.0.203.190:35915)
0619 12:08:39.392536 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:39.392547 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:39.392585 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:08:39.393154 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:39.393165 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:39.393241 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:39.393417 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:39.394256 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.395698 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.396651 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.397286 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.397418 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:39.400562 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:39.400568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:39.400569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:39.400591 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:39.402537 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:39.402923 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:39.402927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:39.402928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:39.403043 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:39.403252 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:39.403253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:39.403253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:39.403638 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:39.403804 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:39.403910 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 12:08:39.404078 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
I20260619 12:08:39.425273 17879 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:39.426164 17879 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:39.450630 17879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15: Generated new TSK 0
I20260619 12:08:39.450884 17879 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:08:39.486337 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35915
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:39.604357 17887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:39.604636 17887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:39.604666 17887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:39.606166 17887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:39.606245 17887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:39.606297 17887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:39.609473 17887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:39.609761 17887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:39.609980 17887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:39.615700 17895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:39.615670 17892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:39.615682 17893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:39.616083 17887 server_base.cc:1034] running on GCE node
I20260619 12:08:39.616328 17887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:39.616559 17887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:39.617700 17887 hybrid_clock.cc:648] HybridClock initialized: now 1781870919617686 us; error 38 us; skew 500 ppm
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:39.620566 17887 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:08:39.621116 17887 webserver.cc:460] Webserver started at http://127.0.203.129:40817/ using document root <none> and password file <none>
I20260619 12:08:39.621438 17887 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:39.621507 17887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:39.621620 17887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:39.622697 17887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "bf69a905682bddb5864a9c148d230e76"
server_key_iv: "93026b35012ae840c1de3a1fe5658c97"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.623023 17887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "bf69a905682bddb5864a9c148d230e76"
server_key_iv: "93026b35012ae840c1de3a1fe5658c97"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.624522 17887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:39.625350 17902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.625594 17887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:39.625684 17887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "bf69a905682bddb5864a9c148d230e76"
server_key_iv: "93026b35012ae840c1de3a1fe5658c97"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.625748 17887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:39.643600 17887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:39.645874 17887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:39.646044 17887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:39.646286 17887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:39.646631 17887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:39.646669 17887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.646699 17887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:39.646715 17887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.653955 17887 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:33319
I20260619 12:08:39.654012 18015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:33319 every 8 connection(s)
I20260619 12:08:39.654400 17887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:39.663936 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 17887
I20260619 12:08:39.664131 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:08:39.664350 814 external_mini_cluster.cc:1428] Setting key 9543832f4201f79fac60b63ea709245c
I20260619 12:08:39.666559 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:39.655392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:41209 (local address 127.0.203.190:35915)
0619 12:08:39.655470 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:39.655473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:39.655713 (+ 240us) server_negotiation.cc:407] Connection header received
0619 12:08:39.656870 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:39.656871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:39.656887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:39.656985 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:39.657859 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.658399 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.659706 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.659942 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.660020 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:39.662881 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:39.662883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:39.662884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:39.662892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:39.664696 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:39.665352 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:39.665354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:39.665354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:39.665466 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:39.665791 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:39.665792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:39.665792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:39.665973 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:39.666051 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:39.666320 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 12:08:39.666449 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:08:39.666644 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:39.655272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35915 (local address 127.0.203.129:41209)
0619 12:08:39.655531 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:39.655583 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:39.656705 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:39.657086 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:39.657091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:39.657456 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:39.657738 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:39.657743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.658536 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.658538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:39.659602 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:39.659606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.660057 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.660058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:39.660202 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:39.660855 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:39.660866 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:39.662726 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:39.664804 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:39.664823 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:39.664827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:39.665234 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:39.665591 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:39.665591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:39.665592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:39.665723 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:39.666039 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:39.666043 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:39.666187 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 12:08:39.666280 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 12:08:39.666448 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":130,"threads_started":1}
I20260619 12:08:39.667264 18016 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35915
I20260619 12:08:39.667416 18016 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:39.667673 18016 heartbeater.cc:511] Master 127.0.203.190:35915 requested a full tablet report, sending...
I20260619 12:08:39.668295 17825 ts_manager.cc:194] Registered new tserver with Master: ed8cf49568cb4ee6850b0a31c3b33aa3 (127.0.203.129:33319)
I20260619 12:08:39.669009 17825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:41209
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:08:39.694974 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35915
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:39.786883 18021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:39.787163 18021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:39.787190 18021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:39.788511 18021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:39.788565 18021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:39.788595 18021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:39.790271 18021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:39.790547 18021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:39.790823 18021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:39.796726 18027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:39.796716 18026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:39.796723 18029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:39.796906 18021 server_base.cc:1034] running on GCE node
I20260619 12:08:39.797343 18021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:39.797622 18021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:39.798794 18021 hybrid_clock.cc:648] HybridClock initialized: now 1781870919798788 us; error 51 us; skew 500 ppm
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:39.802004 18021 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:08:39.802584 18021 webserver.cc:460] Webserver started at http://127.0.203.130:39691/ using document root <none> and password file <none>
I20260619 12:08:39.802839 18021 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:39.802889 18021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:39.803022 18021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:39.804301 18021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "818ba8a1031e48d7bb241dbd1083bb40"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "89f0ffe7e4ca19293c1be74322a75b73"
server_key_iv: "64508e08ada226da1f2ae40cf6b17e93"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.804684 18021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "818ba8a1031e48d7bb241dbd1083bb40"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "89f0ffe7e4ca19293c1be74322a75b73"
server_key_iv: "64508e08ada226da1f2ae40cf6b17e93"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.806178 18021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:08:39.807118 18036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.807348 18021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:39.807483 18021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "818ba8a1031e48d7bb241dbd1083bb40"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "89f0ffe7e4ca19293c1be74322a75b73"
server_key_iv: "64508e08ada226da1f2ae40cf6b17e93"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.807574 18021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:39.828989 18021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:39.831544 18021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:39.831709 18021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:39.831977 18021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:39.832325 18021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:39.832360 18021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.832394 18021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:39.832410 18021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.839578 18021 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:42429
I20260619 12:08:39.839658 18149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:42429 every 8 connection(s)
I20260619 12:08:39.840015 18021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:39.841619 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18021
I20260619 12:08:39.841723 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:08:39.841925 814 external_mini_cluster.cc:1428] Setting key a3dad5cdcee033031631cd69088d7159
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
I20260619 12:08:39.852070 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:39.840870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:44839 (local address 127.0.203.190:35915)
0619 12:08:39.840925 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:39.840928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:39.841149 (+ 221us) server_negotiation.cc:407] Connection header received
0619 12:08:39.842302 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:39.842303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:39.842317 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:39.842367 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:39.843061 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.843637 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.845079 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.845313 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.845392 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:39.848747 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:39.848749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:39.848749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:39.848759 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:39.850191 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:39.850824 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:39.850825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:39.850826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:39.850889 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:39.851237 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:39.851239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:39.851239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:39.851444 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:39.851518 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:39.851893 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 12:08:39.851986 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:08:39.852214 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:39.840829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35915 (local address 127.0.203.130:44839)
0619 12:08:39.841010 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:39.841052 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:39.842183 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:39.842444 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:39.842448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:39.842763 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:39.842971 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:39.842975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.843724 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.843725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:39.844908 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:39.844911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:39.845416 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:39.845417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:39.845565 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:39.846206 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:39.846225 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:39.848578 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:39.850320 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:39.850336 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:39.850339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:39.850706 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:39.850972 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:39.850973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:39.850974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:39.851124 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:39.851523 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:39.851530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:39.851728 (+ 198us) client_negotiation.cc:769] Sending connection context
0619 12:08:39.851859 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 12:08:39.852013 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260619 12:08:39.852934 18150 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35915
I20260619 12:08:39.853044 18150 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:39.853227 18150 heartbeater.cc:511] Master 127.0.203.190:35915 requested a full tablet report, sending...
I20260619 12:08:39.853621 17825 ts_manager.cc:194] Registered new tserver with Master: 818ba8a1031e48d7bb241dbd1083bb40 (127.0.203.130:42429)
I20260619 12:08:39.854065 17825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:44839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:08:39.876685 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35915
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:39.969586 18155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:39.969844 18155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:39.969883 18155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:39.971253 18155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:39.971315 18155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:39.971346 18155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:39.973150 18155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:35915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:39.973555 18155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:39.973853 18155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:39.979688 18160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:39.979691 18163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:39.979794 18161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:39.980101 18155 server_base.cc:1034] running on GCE node
I20260619 12:08:39.980368 18155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:39.980650 18155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:39.981864 18155 hybrid_clock.cc:648] HybridClock initialized: now 1781870919981841 us; error 41 us; skew 500 ppm
Jun 19 12:08:39 dist-test-slave-44vj krb5kdc[17780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:39.985077 18155 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:08:39.985683 18155 webserver.cc:460] Webserver started at http://127.0.203.131:45307/ using document root <none> and password file <none>
I20260619 12:08:39.985913 18155 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:39.985957 18155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:39.986070 18155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:39.987471 18155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "72ae934b38bb4ecf88fbbe09bdef09c7"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "3cc9556eb14a46e7fa829df7476d446a"
server_key_iv: "9ca9258116d5e58d1b0471c7a6197431"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.987844 18155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "72ae934b38bb4ecf88fbbe09bdef09c7"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "3cc9556eb14a46e7fa829df7476d446a"
server_key_iv: "9ca9258116d5e58d1b0471c7a6197431"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.989426 18155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:08:39.990418 18170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:39.990727 18155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260619 12:08:39.990828 18155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "72ae934b38bb4ecf88fbbe09bdef09c7"
format_stamp: "Formatted at 2026-06-19 12:08:39 on dist-test-slave-44vj"
server_key: "3cc9556eb14a46e7fa829df7476d446a"
server_key_iv: "9ca9258116d5e58d1b0471c7a6197431"
server_key_version: "encryptionkey@0"
I20260619 12:08:39.990911 18155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:40.002607 18155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:40.004987 18155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:40.005128 18155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:40.005416 18155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:40.005757 18155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:40.005792 18155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.005815 18155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:40.005838 18155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.014227 18155 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:33893
I20260619 12:08:40.014295 18283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:33893 every 8 connection(s)
I20260619 12:08:40.015049 18155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:08:40 dist-test-slave-44vj krb5kdc[17780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:40.023423 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18155
I20260619 12:08:40.023535 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:08:40.023734 814 external_mini_cluster.cc:1428] Setting key 16e37f449b606ccdd0a8b7dd6d476e40
I20260619 12:08:40.027908 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.016193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:55769 (local address 127.0.203.190:35915)
0619 12:08:40.016295 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.016298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.016590 (+ 292us) server_negotiation.cc:407] Connection header received
0619 12:08:40.017546 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.017548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.017564 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.017650 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:40.018382 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.018969 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.020260 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.020488 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.020601 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.023704 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:40.023706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:40.023706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:40.023718 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:40.025692 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:40.026418 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:40.026419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:40.026420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:40.026503 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:40.026941 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:40.026943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:40.026943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:40.027114 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:40.027193 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.027695 (+ 502us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.027813 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:08:40.028319 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.016133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:35915 (local address 127.0.203.131:55769)
0619 12:08:40.016336 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.016396 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.017404 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.017727 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.017731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.018047 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:40.018265 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.018269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.019122 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.019123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.020144 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.020148 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.020640 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.020641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.020772 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.021409 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:40.021418 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:40.023528 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:40.025825 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:40.025836 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:40.025838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:40.026293 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:40.026628 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:40.026630 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:40.026630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:40.026828 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:40.027234 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:40.027242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:40.027504 (+ 262us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.027696 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.027963 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260619 12:08:40.028998 18284 heartbeater.cc:347] Connected to a master server at 127.0.203.190:35915
I20260619 12:08:40.029109 18284 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:40.029333 18284 heartbeater.cc:511] Master 127.0.203.190:35915 requested a full tablet report, sending...
I20260619 12:08:40.029738 17825 ts_manager.cc:194] Registered new tserver with Master: 72ae934b38bb4ecf88fbbe09bdef09c7 (127.0.203.131:33893)
I20260619 12:08:40.030114 17825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:55769
I20260619 12:08:40.036855 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:08:40.046274 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.038524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.0.203.190:35915)
0619 12:08:40.038598 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.038600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.038740 (+ 140us) server_negotiation.cc:407] Connection header received
0619 12:08:40.038800 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.038801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.038814 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.038898 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:40.039679 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.040200 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.041093 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.041308 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.041383 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.042950 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:40.042951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:40.042952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:40.042961 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:40.044786 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:40.045286 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:40.045288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:40.045289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:40.045376 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:40.045616 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:40.045617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:40.045617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:40.045837 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:40.045937 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.046093 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.046185 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:08:40.048663 17825 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:08:40.050770 17825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:08:40.058691 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.054497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.0.203.130:42429)
0619 12:08:40.054693 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.054698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.054710 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:08:40.054863 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.054865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.054885 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.054988 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.055312 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.056101 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.057371 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.058155 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.058239 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.058304 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.058535 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.058586 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":100,"threads_started":1}
I20260619 12:08:40.058704 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.054251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:42429 (local address 127.0.0.1:45622)
0619 12:08:40.054488 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.054549 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.054673 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.055104 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.055108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.055111 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.055204 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.055207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.056210 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.056211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.057260 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.057264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.058254 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.058254 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.058383 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.058408 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.058502 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.058584 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":110,"threads_started":1}
I20260619 12:08:40.059432 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.054532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:33893 (local address 127.0.0.1:33544)
0619 12:08:40.054798 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.054809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.054854 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.055397 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.055398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.055401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.055439 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.055440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.056949 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.056950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.058065 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.058068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.059144 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.059145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.059211 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.059215 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.059269 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.059326 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260619 12:08:40.059451 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.054815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.0.203.131:33893)
0619 12:08:40.055027 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.055030 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.055041 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:40.055180 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.055183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.055201 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.055280 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.055550 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.056786 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.058180 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.059056 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.059126 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.059194 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.059288 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.059339 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":96,"threads_started":1}
I20260619 12:08:40.059777 18084 tablet_service.cc:1467] Processing CreateTablet for tablet 4f917c76532f4385b3ab0b3d00a47988 (DEFAULT_TABLE table=test-table [id=66e10c2faaa0470dbb58820836e09116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:40.060236 18084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f917c76532f4385b3ab0b3d00a47988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:40.060300 18218 tablet_service.cc:1467] Processing CreateTablet for tablet 4f917c76532f4385b3ab0b3d00a47988 (DEFAULT_TABLE table=test-table [id=66e10c2faaa0470dbb58820836e09116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:40.060477 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.054444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.0.203.129:33319)
0619 12:08:40.054803 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.054807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.054917 (+ 110us) server_negotiation.cc:407] Connection header received
0619 12:08:40.054977 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.054980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.054998 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.055072 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.055412 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.056767 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.058418 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.059402 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.059476 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.060061 (+ 585us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.060310 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.060390 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":99,"threads_started":1}
I20260619 12:08:40.060714 18218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f917c76532f4385b3ab0b3d00a47988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:40.060773 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.054282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33319 (local address 127.0.0.1:55534)
0619 12:08:40.054782 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.054792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.054854 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.055214 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.055215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.055218 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.055302 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.055306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.056921 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.056922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.058289 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.058293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.060103 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.060104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.060184 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.060189 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.060257 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.060420 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":188,"thread_start_us":129,"threads_started":1}
I20260619 12:08:40.063855 18301 tablet_bootstrap.cc:492] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Bootstrap starting.
I20260619 12:08:40.064218 18302 tablet_bootstrap.cc:492] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Bootstrap starting.
I20260619 12:08:40.064142 17950 tablet_service.cc:1467] Processing CreateTablet for tablet 4f917c76532f4385b3ab0b3d00a47988 (DEFAULT_TABLE table=test-table [id=66e10c2faaa0470dbb58820836e09116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:40.064518 17950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f917c76532f4385b3ab0b3d00a47988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:40.064963 18301 tablet_bootstrap.cc:654] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:40.065135 18302 tablet_bootstrap.cc:654] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:40.065534 18301 log.cc:824] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:40.065758 18302 log.cc:824] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:40.066576 18301 tablet_bootstrap.cc:492] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: No bootstrap required, opened a new log
I20260619 12:08:40.066675 18301 ts_tablet_manager.cc:1397] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:08:40.068084 18305 tablet_bootstrap.cc:492] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3: Bootstrap starting.
I20260619 12:08:40.068378 18302 tablet_bootstrap.cc:492] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: No bootstrap required, opened a new log
I20260619 12:08:40.068468 18302 ts_tablet_manager.cc:1397] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:08:40.069080 18305 tablet_bootstrap.cc:654] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:40.069271 18301 raft_consensus.cc:348] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.069535 18301 raft_consensus.cc:374] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:40.069586 18301 raft_consensus.cc:729] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818ba8a1031e48d7bb241dbd1083bb40, State: Initialized, Role: FOLLOWER
I20260619 12:08:40.069759 18305 log.cc:824] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:40.069743 18301 consensus_queue.cc:260] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.070154 18301 ts_tablet_manager.cc:1428] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 12:08:40.070235 18150 heartbeater.cc:503] Master 127.0.203.190:35915 was elected leader, sending a full tablet report...
I20260619 12:08:40.070513 18302 raft_consensus.cc:348] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.070731 18302 raft_consensus.cc:374] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:40.070768 18302 raft_consensus.cc:729] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ae934b38bb4ecf88fbbe09bdef09c7, State: Initialized, Role: FOLLOWER
I20260619 12:08:40.071128 18302 consensus_queue.cc:260] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.071550 18302 ts_tablet_manager.cc:1428] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:08:40.071635 18305 tablet_bootstrap.cc:492] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3: No bootstrap required, opened a new log
I20260619 12:08:40.071693 18284 heartbeater.cc:503] Master 127.0.203.190:35915 was elected leader, sending a full tablet report...
I20260619 12:08:40.071729 18305 ts_tablet_manager.cc:1397] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:08:40.073756 18305 raft_consensus.cc:348] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.074007 18305 raft_consensus.cc:374] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:40.074044 18305 raft_consensus.cc:729] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed8cf49568cb4ee6850b0a31c3b33aa3, State: Initialized, Role: FOLLOWER
I20260619 12:08:40.074206 18305 consensus_queue.cc:260] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.074563 18305 ts_tablet_manager.cc:1428] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:08:40.074638 18016 heartbeater.cc:503] Master 127.0.203.190:35915 was elected leader, sending a full tablet report...
W20260619 12:08:40.090749 18151 tablet.cc:2369] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:40.096695 18309 raft_consensus.cc:482] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:40.096860 18309 raft_consensus.cc:504] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.097275 18309 leader_election.cc:288] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 818ba8a1031e48d7bb241dbd1083bb40 (127.0.203.130:42429), 72ae934b38bb4ecf88fbbe09bdef09c7 (127.0.203.131:33893)
I20260619 12:08:40.100653 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.097596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:42429 (local address 127.0.203.129:43583)
0619 12:08:40.097724 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.097738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.097793 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.097939 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.097940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.097943 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.098036 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.098040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.098706 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.098707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.099751 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.099755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.100350 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.100351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.100438 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.100442 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.100488 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.100551 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260619 12:08:40.100648 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.097674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:43583 (local address 127.0.203.130:42429)
0619 12:08:40.097730 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.097733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.097771 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:08:40.097865 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.097867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.097877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.097924 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.098096 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.098624 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.099853 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.100273 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.100347 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.100360 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.100504 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.100552 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:08:40.100847 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.097667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:53243 (local address 127.0.203.131:33893)
0619 12:08:40.097757 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.097759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.098013 (+ 254us) server_negotiation.cc:407] Connection header received
0619 12:08:40.098097 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.098098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.098108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.098163 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.098323 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.098929 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.100027 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.100456 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.100528 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.100537 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.100709 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.100755 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 12:08:40.101035 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.097596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:33893 (local address 127.0.203.129:53243)
0619 12:08:40.097896 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.097909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.098036 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.098170 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.098171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.098172 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.098267 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.098270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.099027 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.099028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.099938 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.099943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.100573 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.100574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.100641 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.100645 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.100691 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.100893 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":64,"thread_start_us":145,"threads_started":1}
I20260619 12:08:40.101130 18104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f917c76532f4385b3ab0b3d00a47988" candidate_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "818ba8a1031e48d7bb241dbd1083bb40" is_pre_election: true
I20260619 12:08:40.101358 18104 raft_consensus.cc:2454] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed8cf49568cb4ee6850b0a31c3b33aa3 in term 0.
I20260619 12:08:40.101424 18238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f917c76532f4385b3ab0b3d00a47988" candidate_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" is_pre_election: true
I20260619 12:08:40.101621 18238 raft_consensus.cc:2454] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed8cf49568cb4ee6850b0a31c3b33aa3 in term 0.
I20260619 12:08:40.101648 17904 leader_election.cc:302] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 818ba8a1031e48d7bb241dbd1083bb40, ed8cf49568cb4ee6850b0a31c3b33aa3; no voters:
I20260619 12:08:40.101814 18309 raft_consensus.cc:2781] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:08:40.101877 18309 raft_consensus.cc:482] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:08:40.101894 18309 raft_consensus.cc:3037] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:40.102887 18309 raft_consensus.cc:504] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.103036 18309 leader_election.cc:288] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [CANDIDATE]: Term 1 election: Requested vote from peers 818ba8a1031e48d7bb241dbd1083bb40 (127.0.203.130:42429), 72ae934b38bb4ecf88fbbe09bdef09c7 (127.0.203.131:33893)
I20260619 12:08:40.103350 18104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f917c76532f4385b3ab0b3d00a47988" candidate_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "818ba8a1031e48d7bb241dbd1083bb40"
I20260619 12:08:40.103315 18238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f917c76532f4385b3ab0b3d00a47988" candidate_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7"
I20260619 12:08:40.103483 18104 raft_consensus.cc:3037] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:40.103507 18238 raft_consensus.cc:3037] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:40.104351 18104 raft_consensus.cc:2454] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed8cf49568cb4ee6850b0a31c3b33aa3 in term 1.
I20260619 12:08:40.104405 18238 raft_consensus.cc:2454] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed8cf49568cb4ee6850b0a31c3b33aa3 in term 1.
I20260619 12:08:40.104571 17904 leader_election.cc:302] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 818ba8a1031e48d7bb241dbd1083bb40, ed8cf49568cb4ee6850b0a31c3b33aa3; no voters:
I20260619 12:08:40.104768 18309 raft_consensus.cc:2781] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:40.105007 18309 raft_consensus.cc:686] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [term 1 LEADER]: Becoming Leader. State: Replica: ed8cf49568cb4ee6850b0a31c3b33aa3, State: Running, Role: LEADER
I20260619 12:08:40.105126 18309 consensus_queue.cc:237] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } }
I20260619 12:08:40.106218 17825 catalog_manager.cc:5495] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed8cf49568cb4ee6850b0a31c3b33aa3 (127.0.203.129). New cstate: current_term: 1 leader_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8cf49568cb4ee6850b0a31c3b33aa3" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 } health_report { overall_health: UNKNOWN } } }
I20260619 12:08:40.119859 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.117318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.0.203.129:33319)
0619 12:08:40.117398 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.117400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.117409 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:08:40.117454 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.117455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.117476 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.117568 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:40.117725 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.118243 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.119002 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.119201 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.119260 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.119469 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:40.119604 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:08:40.119688 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.119706 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.119732 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:08:40.122234 18238 raft_consensus.cc:1261] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 1 FOLLOWER]: Refusing update from remote peer ed8cf49568cb4ee6850b0a31c3b33aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:40.122290 18104 raft_consensus.cc:1261] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 1 FOLLOWER]: Refusing update from remote peer ed8cf49568cb4ee6850b0a31c3b33aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:40.122684 18309 consensus_queue.cc:1035] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ae934b38bb4ecf88fbbe09bdef09c7" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 33893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:40.122879 18311 consensus_queue.cc:1035] T 4f917c76532f4385b3ab0b3d00a47988 P ed8cf49568cb4ee6850b0a31c3b33aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "818ba8a1031e48d7bb241dbd1083bb40" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:40.126533 18314 mvcc.cc:205] Tried to move back new op lower bound from 7298543288817364992 to 7298543288751394816. Current Snapshot: MvccSnapshot[applied={T|T < 7298543288817364992 or (T in {7298543288817364992})}]
I20260619 12:08:40.126753 18316 mvcc.cc:205] Tried to move back new op lower bound from 7298543288817364992 to 7298543288751394816. Current Snapshot: MvccSnapshot[applied={T|T < 7298543288817364992}]
I20260619 12:08:40.129019 17824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:40.129129 17824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:40.129971 17824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3456f177f374c94a3f38804e8e97f15: Sending DeleteTablet for 3 replicas of tablet 4f917c76532f4385b3ab0b3d00a47988
I20260619 12:08:40.130327 18084 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f917c76532f4385b3ab0b3d00a47988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:40 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:45622
I20260619 12:08:40.130330 17950 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f917c76532f4385b3ab0b3d00a47988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:40 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:55534
I20260619 12:08:40.130360 18218 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f917c76532f4385b3ab0b3d00a47988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:40 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:33544
I20260619 12:08:40.130587 18326 tablet_replica.cc:330] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: stopping tablet replica
I20260619 12:08:40.130802 18326 raft_consensus.cc:2229] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:40.130940 18326 raft_consensus.cc:2258] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:40.130954 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 17887
I20260619 12:08:40.131979 18327 tablet_replica.cc:330] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: stopping tablet replica
I20260619 12:08:40.132112 18327 raft_consensus.cc:2229] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:40.132213 18327 raft_consensus.cc:2258] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:40.132762 18327 ts_tablet_manager.cc:1905] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:40.133033 18326 ts_tablet_manager.cc:1905] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:40.134410 18327 ts_tablet_manager.cc:1918] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:40.134509 18327 log.cc:1196] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/4f917c76532f4385b3ab0b3d00a47988
I20260619 12:08:40.134511 18326 ts_tablet_manager.cc:1918] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:40.134606 18326 log.cc:1196] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/4f917c76532f4385b3ab0b3d00a47988
I20260619 12:08:40.134795 18327 ts_tablet_manager.cc:1939] T 4f917c76532f4385b3ab0b3d00a47988 P 72ae934b38bb4ecf88fbbe09bdef09c7: Deleting consensus metadata
I20260619 12:08:40.134865 18326 ts_tablet_manager.cc:1939] T 4f917c76532f4385b3ab0b3d00a47988 P 818ba8a1031e48d7bb241dbd1083bb40: Deleting consensus metadata
I20260619 12:08:40.135257 17809 catalog_manager.cc:4841] TS 72ae934b38bb4ecf88fbbe09bdef09c7 (127.0.203.131:33893): tablet 4f917c76532f4385b3ab0b3d00a47988 (table test-table [id=66e10c2faaa0470dbb58820836e09116]) successfully deleted
I20260619 12:08:40.135353 17810 catalog_manager.cc:4841] TS 818ba8a1031e48d7bb241dbd1083bb40 (127.0.203.130:42429): tablet 4f917c76532f4385b3ab0b3d00a47988 (table test-table [id=66e10c2faaa0470dbb58820836e09116]) successfully deleted
W20260619 12:08:40.138687 17809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:33319 (error 108)
I20260619 12:08:40.139448 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18021
I20260619 12:08:40.139534 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.139339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33319 (local address 127.0.0.1:55554)
0619 12:08:40.139441 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.139495 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 12:08:40.139647 17809 catalog_manager.cc:4584] TS ed8cf49568cb4ee6850b0a31c3b33aa3 (127.0.203.129:33319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f917c76532f4385b3ab0b3d00a47988: Network error: Client connection negotiation failed: client connection to 127.0.203.129:33319: connect: Connection refused (error 111)
I20260619 12:08:40.145129 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18155
I20260619 12:08:40.150449 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 17793
2026-06-19T12:08:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2008 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T12:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:40Z Disabled control of system clock
I20260619 12:08:40.180339 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:39081
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:39081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:40.272333 18332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:40.272640 18332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:40.272670 18332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:40.274067 18332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:40.274130 18332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:40.274142 18332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:40.274154 18332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:40.274165 18332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:40.275828 18332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:39081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:39081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:40.276127 18332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:40.276451 18332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:40.281807 18337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:40.281822 18338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:40.282073 18332 server_base.cc:1034] running on GCE node
W20260619 12:08:40.281822 18340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:40.282451 18332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:40.282802 18332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:40.283975 18332 hybrid_clock.cc:648] HybridClock initialized: now 1781870920283961 us; error 36 us; skew 500 ppm
I20260619 12:08:40.285920 18332 webserver.cc:460] Webserver started at http://127.0.203.190:41195/ using document root <none> and password file <none>
I20260619 12:08:40.286237 18332 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:40.286310 18332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:40.286463 18332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:40.288187 18332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "dcd37ea5d4fb4e7f8921efc52826a56e"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "c875dabbe9f3ae9d4c8c14be1a4b1644"
server_key_iv: "b902a4d6c577f1e41dd0524086a7761d"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.288647 18332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "dcd37ea5d4fb4e7f8921efc52826a56e"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "c875dabbe9f3ae9d4c8c14be1a4b1644"
server_key_iv: "b902a4d6c577f1e41dd0524086a7761d"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.290599 18332 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:08:40.292256 18346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.294250 18332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 12:08:40.294410 18332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "dcd37ea5d4fb4e7f8921efc52826a56e"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "c875dabbe9f3ae9d4c8c14be1a4b1644"
server_key_iv: "b902a4d6c577f1e41dd0524086a7761d"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.294535 18332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:40.316946 18332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:40.317404 18332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:40.317569 18332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:40.323119 18332 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:39081
I20260619 12:08:40.323173 18398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:39081 every 8 connection(s)
I20260619 12:08:40.323727 18332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:40.325605 18399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:40.326542 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18332
I20260619 12:08:40.326656 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:40.326874 814 external_mini_cluster.cc:1428] Setting key e25ff091c3d984b766a63e9430613c6e
I20260619 12:08:40.329572 18399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e: Bootstrap starting.
I20260619 12:08:40.330335 18399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:40.330632 18399 log.cc:824] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:40.331349 18399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e: No bootstrap required, opened a new log
I20260619 12:08:40.333874 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.327720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.0.203.190:39081)
0619 12:08:40.328012 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.328023 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.328062 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:08:40.328635 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.328645 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.328753 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.328932 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:40.329212 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.330527 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.331467 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.332078 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.332222 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.333047 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:40.333052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:40.333052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:40.333277 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:40.333326 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:40.333344 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:40.333426 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.333503 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.333606 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":136,"threads_started":1}
I20260619 12:08:40.333945 18399 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39081 } }
I20260619 12:08:40.334116 18399 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:40.334152 18399 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcd37ea5d4fb4e7f8921efc52826a56e, State: Initialized, Role: FOLLOWER
I20260619 12:08:40.334271 18399 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39081 } }
I20260619 12:08:40.334332 18399 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:40.334358 18399 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:40.334414 18399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:40.335042 18399 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39081 } }
I20260619 12:08:40.335146 18399 leader_election.cc:302] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcd37ea5d4fb4e7f8921efc52826a56e; no voters:
I20260619 12:08:40.335381 18399 leader_election.cc:288] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:40.335464 18404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:40.335695 18404 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [term 1 LEADER]: Becoming Leader. State: Replica: dcd37ea5d4fb4e7f8921efc52826a56e, State: Running, Role: LEADER
I20260619 12:08:40.335731 18399 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:40.335812 18404 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39081 } }
I20260619 12:08:40.336256 18405 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39081 } } }
I20260619 12:08:40.336265 18406 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcd37ea5d4fb4e7f8921efc52826a56e. Latest consensus state: current_term: 1 leader_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd37ea5d4fb4e7f8921efc52826a56e" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39081 } } }
I20260619 12:08:40.336374 18405 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:40.336380 18406 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:40.336645 18413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:40.337270 18413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:40.339145 18413 catalog_manager.cc:1347] Generated new cluster ID: 23009d480879461ea5a20f1fa972de9c
I20260619 12:08:40.339214 18413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:40.347293 18413 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:40.348042 18413 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:40.355662 18413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e: Generated new TSK 0
I20260619 12:08:40.355921 18413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:40.366001 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:39081
--builtin_ntp_servers=127.0.203.148:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:40.500291 18423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:40.500685 18423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:40.500726 18423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:40.502749 18423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:40.502832 18423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:40.502874 18423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:40.505149 18423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:39081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:40.505497 18423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:40.505908 18423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:40.511886 18431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:40.511890 18429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:40.511879 18428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:40.512194 18423 server_base.cc:1034] running on GCE node
I20260619 12:08:40.512583 18423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:40.512934 18423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:40.514142 18423 hybrid_clock.cc:648] HybridClock initialized: now 1781870920514115 us; error 43 us; skew 500 ppm
I20260619 12:08:40.515820 18423 webserver.cc:460] Webserver started at http://127.0.203.129:43051/ using document root <none> and password file <none>
I20260619 12:08:40.516130 18423 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:40.516194 18423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:40.516330 18423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:40.517946 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "26a5fa6f16836af2e4bab73ff5a120c4"
server_key_iv: "578d2a92116fb18a63d4b7d96818c479"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.518364 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "26a5fa6f16836af2e4bab73ff5a120c4"
server_key_iv: "578d2a92116fb18a63d4b7d96818c479"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.520363 18423 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 12:08:40.521461 18437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.521739 18423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:40.521839 18423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "26a5fa6f16836af2e4bab73ff5a120c4"
server_key_iv: "578d2a92116fb18a63d4b7d96818c479"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.521919 18423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:40.558111 18423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:40.558527 18423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:40.558655 18423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:40.558917 18423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:40.559351 18423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:40.559429 18423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.559454 18423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:40.559469 18423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.566885 18423 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:39431
I20260619 12:08:40.566969 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:39431 every 8 connection(s)
I20260619 12:08:40.567456 18423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:08:40.574127 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.568391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:42515 (local address 127.0.203.190:39081)
0619 12:08:40.568456 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.568458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.568903 (+ 445us) server_negotiation.cc:407] Connection header received
0619 12:08:40.569516 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.569518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.569536 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.569622 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:40.570447 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.570985 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.572327 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.572565 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.572647 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.573720 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:40.573722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:40.573723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:40.573731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:40.573758 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:40.573764 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:40.573825 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.573978 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.574029 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:08:40.574301 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.568423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:39081 (local address 127.0.203.129:42515)
0619 12:08:40.568708 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.568754 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.569395 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.569703 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.569708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.569719 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:40.570326 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.570330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.571145 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.571147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.572196 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.572200 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.572698 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.572700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.572853 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.573502 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:40.573503 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:40.573540 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:40.573582 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:40.573852 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:40.573859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:40.573862 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.573940 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.574065 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":155,"threads_started":1}
I20260619 12:08:40.575040 18551 heartbeater.cc:347] Connected to a master server at 127.0.203.190:39081
I20260619 12:08:40.575201 18551 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:40.575461 18551 heartbeater.cc:511] Master 127.0.203.190:39081 requested a full tablet report, sending...
I20260619 12:08:40.576197 18363 ts_manager.cc:194] Registered new tserver with Master: a3e7cb6d3e9b498f8ede9e7c93b963ab (127.0.203.129:39431)
I20260619 12:08:40.576412 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18423
I20260619 12:08:40.576512 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:08:40.576694 814 external_mini_cluster.cc:1428] Setting key 0c8fd0453ca940d8ce909d15df8b0aee
I20260619 12:08:40.577044 18363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.129:42515
I20260619 12:08:40.577827 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:39081
--builtin_ntp_servers=127.0.203.148:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:40.669692 18554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:40.669961 18554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:40.669986 18554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:40.671344 18554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:40.671435 18554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:40.671465 18554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:40.673259 18554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:39081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:40.673573 18554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:40.673926 18554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:40.679657 18560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:40.679697 18559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:40.679916 18554 server_base.cc:1034] running on GCE node
W20260619 12:08:40.679657 18562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:40.680330 18554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:40.680619 18554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:40.681800 18554 hybrid_clock.cc:648] HybridClock initialized: now 1781870920681776 us; error 49 us; skew 500 ppm
I20260619 12:08:40.683290 18554 webserver.cc:460] Webserver started at http://127.0.203.130:43849/ using document root <none> and password file <none>
I20260619 12:08:40.683555 18554 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:40.683609 18554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:40.683743 18554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:40.684856 18554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "347cf481f19a4badaf2b4c61cf82531e"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "72f941066082c678dad6dfdd3ffb8f27"
server_key_iv: "8ae35b543f8824e8aaf050498d2e77ba"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.685200 18554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "347cf481f19a4badaf2b4c61cf82531e"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "72f941066082c678dad6dfdd3ffb8f27"
server_key_iv: "8ae35b543f8824e8aaf050498d2e77ba"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.686827 18554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:08:40.687820 18568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.688138 18554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:40.688277 18554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "347cf481f19a4badaf2b4c61cf82531e"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "72f941066082c678dad6dfdd3ffb8f27"
server_key_iv: "8ae35b543f8824e8aaf050498d2e77ba"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.688364 18554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:40.698328 18554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:40.698662 18554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:40.698792 18554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:40.699030 18554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:40.699425 18554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:40.699470 18554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.699502 18554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:40.699525 18554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.706918 18554 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:43647
I20260619 12:08:40.707078 18681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:43647 every 8 connection(s)
I20260619 12:08:40.707350 18554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:40.713635 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.708375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:42729 (local address 127.0.203.190:39081)
0619 12:08:40.708454 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.708456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.708720 (+ 264us) server_negotiation.cc:407] Connection header received
0619 12:08:40.709252 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.709254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.709267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.709324 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:40.710006 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.710552 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.712007 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.712227 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.712293 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.713344 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:40.713345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:40.713346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:40.713351 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:40.713368 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:40.713372 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:40.713414 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.713511 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.713547 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:08:40.713773 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18554
I20260619 12:08:40.713879 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:08:40.713903 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.708353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:39081 (local address 127.0.203.130:42729)
0619 12:08:40.708585 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.708627 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.709149 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.709399 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.709403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.709412 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:40.709916 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.709920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.710686 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.710688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.711884 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.711892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.712358 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.712359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.712513 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.713187 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:40.713187 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:40.713221 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:40.713249 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:40.713440 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:40.713444 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:40.713447 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.713506 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.713653 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
I20260619 12:08:40.714094 814 external_mini_cluster.cc:1428] Setting key 58d36b2c4aa8ec52f0fcf5f715d1a50d
I20260619 12:08:40.714485 18682 heartbeater.cc:347] Connected to a master server at 127.0.203.190:39081
I20260619 12:08:40.714632 18682 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:40.714886 18682 heartbeater.cc:511] Master 127.0.203.190:39081 requested a full tablet report, sending...
I20260619 12:08:40.715186 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:39081
--builtin_ntp_servers=127.0.203.148:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:08:40.715339 18362 ts_manager.cc:194] Registered new tserver with Master: 347cf481f19a4badaf2b4c61cf82531e (127.0.203.130:43647)
I20260619 12:08:40.715768 18362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.130:42729
W20260619 12:08:40.806835 18685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:40.807142 18685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:40.807168 18685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:40.808600 18685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:40.808661 18685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:40.808692 18685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:40.810189 18685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:39081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:40.810470 18685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:40.810840 18685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:40.816464 18690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:40.816478 18691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:40.816576 18685 server_base.cc:1034] running on GCE node
W20260619 12:08:40.816443 18693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:40.816977 18685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:40.817298 18685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:40.818497 18685 hybrid_clock.cc:648] HybridClock initialized: now 1781870920818462 us; error 59 us; skew 500 ppm
I20260619 12:08:40.820076 18685 webserver.cc:460] Webserver started at http://127.0.203.131:41497/ using document root <none> and password file <none>
I20260619 12:08:40.820317 18685 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:40.820366 18685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:40.820468 18685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:40.821631 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "0ec5ea4d409e4b9a85ebb3379069a85f"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "5e4b33e965f396927e9c81683ef54e68"
server_key_iv: "baaa3cdd9b54651332432929d0fad778"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.821967 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "0ec5ea4d409e4b9a85ebb3379069a85f"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "5e4b33e965f396927e9c81683ef54e68"
server_key_iv: "baaa3cdd9b54651332432929d0fad778"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.823747 18685 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:08:40.824821 18699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.825057 18685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:40.825352 18685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "0ec5ea4d409e4b9a85ebb3379069a85f"
format_stamp: "Formatted at 2026-06-19 12:08:40 on dist-test-slave-44vj"
server_key: "5e4b33e965f396927e9c81683ef54e68"
server_key_iv: "baaa3cdd9b54651332432929d0fad778"
server_key_version: "encryptionkey@0"
I20260619 12:08:40.825471 18685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:40.840708 18685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:40.841243 18685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:40.841401 18685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:40.841665 18685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:40.842048 18685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:40.842090 18685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.842113 18685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:40.842125 18685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:40.848474 18685 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:44911
I20260619 12:08:40.848523 18812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:44911 every 8 connection(s)
I20260619 12:08:40.848863 18685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:08:40.851390 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18685
I20260619 12:08:40.851526 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:08:40.851734 814 external_mini_cluster.cc:1428] Setting key 746119c34fd9bcb854b6ab4214df6442
I20260619 12:08:40.855212 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.849721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:37331 (local address 127.0.203.190:39081)
0619 12:08:40.849806 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.849809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.849983 (+ 174us) server_negotiation.cc:407] Connection header received
0619 12:08:40.850500 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.850501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.850518 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.850609 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:40.851428 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.852226 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.853542 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.853759 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.853830 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.854873 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:40.854875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:40.854875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:40.854879 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:40.854897 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:40.854901 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:40.854954 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.855091 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.855118 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:08:40.855479 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.849666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:39081 (local address 127.0.203.131:37331)
0619 12:08:40.849853 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.849894 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.850373 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.850740 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.850743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.850756 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:40.851283 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.851287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.852374 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.852375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.853421 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.853426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.853868 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.853869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.854010 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.854712 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:40.854713 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:40.854745 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:40.854773 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:40.854997 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:40.855003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:40.855006 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.855087 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.855187 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260619 12:08:40.856032 18813 heartbeater.cc:347] Connected to a master server at 127.0.203.190:39081
I20260619 12:08:40.856168 18813 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:40.856408 18813 heartbeater.cc:511] Master 127.0.203.190:39081 requested a full tablet report, sending...
I20260619 12:08:40.856882 18362 ts_manager.cc:194] Registered new tserver with Master: 0ec5ea4d409e4b9a85ebb3379069a85f (127.0.203.131:44911)
I20260619 12:08:40.857312 18362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.131:37331
I20260619 12:08:40.864078 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:08:40.869092 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.865796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.0.203.190:39081)
0619 12:08:40.865875 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.865877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.866036 (+ 159us) server_negotiation.cc:407] Connection header received
0619 12:08:40.866093 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.866094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.866107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.866162 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:40.866323 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.866856 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.867803 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.868004 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.868076 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.868846 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:40.868848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:40.868848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:40.868853 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:40.868872 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:40.868876 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:40.868927 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.868995 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.869022 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:08:40.870952 18362 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:08:40.872779 18362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:08:40.880455 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.875974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.0.203.131:44911)
0619 12:08:40.876175 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.876180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.876194 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:08:40.876239 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.876241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.876259 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.876342 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.876605 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.877776 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.879435 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.880020 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.880092 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.880201 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.880293 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.880343 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":89,"threads_started":1}
I20260619 12:08:40.880442 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.875582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:44911 (local address 127.0.0.1:60684)
0619 12:08:40.875873 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.875920 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.876001 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.876431 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.876434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.876437 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.876526 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.876529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.877885 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.877886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.879229 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.879233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.880102 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.880103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.880188 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.880202 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.880277 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.880344 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":74,"spinlock_wait_cycles":384,"thread_start_us":169,"threads_started":1}
I20260619 12:08:40.881058 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.875582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39431 (local address 127.0.0.1:44686)
0619 12:08:40.875873 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.875920 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.876001 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.876666 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.876667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.876670 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.876720 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.876721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.878065 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.878067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.879186 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.879190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.880405 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.880406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.880486 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.880492 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.880528 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.880939 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":162,"threads_started":1}
I20260619 12:08:40.881058 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.875680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:43647 (local address 127.0.0.1:55902)
0619 12:08:40.876503 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.876512 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.876576 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.876758 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.876759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.876762 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.876835 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.876837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.877737 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.877738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.879230 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.879233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.880515 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.880516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.880678 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.880684 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.880739 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.880930 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":46,"thread_start_us":114,"threads_started":1}
I20260619 12:08:40.881279 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.876000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55902 (local address 127.0.203.130:43647)
0619 12:08:40.876446 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.876451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.876627 (+ 176us) server_negotiation.cc:407] Connection header received
0619 12:08:40.876671 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.876673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.876687 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.876758 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.876897 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.877659 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.879411 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.880273 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.880365 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.880473 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.881114 (+ 641us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.881178 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":81,"threads_started":1}
I20260619 12:08:40.881620 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.876204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.0.203.129:39431)
0619 12:08:40.876441 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.876446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.876456 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:08:40.876503 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.876505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.876524 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.876651 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.876824 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.877980 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.879444 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.880272 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.880911 (+ 639us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.881474 (+ 563us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.881501 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.881545 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":94,"threads_started":1}
I20260619 12:08:40.881523 18747 tablet_service.cc:1467] Processing CreateTablet for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9 (DEFAULT_TABLE table=test-table [id=9d4af1f0661d484fb09f725aa156b8ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:40.881958 18747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:40.882141 18616 tablet_service.cc:1467] Processing CreateTablet for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9 (DEFAULT_TABLE table=test-table [id=9d4af1f0661d484fb09f725aa156b8ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:40.882362 18485 tablet_service.cc:1467] Processing CreateTablet for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9 (DEFAULT_TABLE table=test-table [id=9d4af1f0661d484fb09f725aa156b8ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:40.882582 18616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:40.882668 18485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:40.885071 18830 tablet_bootstrap.cc:492] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Bootstrap starting.
I20260619 12:08:40.885818 18830 tablet_bootstrap.cc:654] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:40.886108 18830 log.cc:824] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:40.886461 18831 tablet_bootstrap.cc:492] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: Bootstrap starting.
I20260619 12:08:40.886983 18833 tablet_bootstrap.cc:492] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab: Bootstrap starting.
I20260619 12:08:40.887223 18831 tablet_bootstrap.cc:654] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:40.887394 18830 tablet_bootstrap.cc:492] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: No bootstrap required, opened a new log
I20260619 12:08:40.887502 18830 ts_tablet_manager.cc:1397] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:40.887570 18831 log.cc:824] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:40.888062 18833 tablet_bootstrap.cc:654] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:40.888310 18831 tablet_bootstrap.cc:492] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: No bootstrap required, opened a new log
I20260619 12:08:40.888379 18831 ts_tablet_manager.cc:1397] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:08:40.888588 18833 log.cc:824] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:40.889652 18833 tablet_bootstrap.cc:492] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab: No bootstrap required, opened a new log
I20260619 12:08:40.889787 18833 ts_tablet_manager.cc:1397] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:08:40.890563 18830 raft_consensus.cc:348] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.890785 18830 raft_consensus.cc:374] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:40.890823 18830 raft_consensus.cc:729] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec5ea4d409e4b9a85ebb3379069a85f, State: Initialized, Role: FOLLOWER
I20260619 12:08:40.890967 18830 consensus_queue.cc:260] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.890933 18831 raft_consensus.cc:348] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.891114 18831 raft_consensus.cc:374] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:40.891157 18831 raft_consensus.cc:729] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347cf481f19a4badaf2b4c61cf82531e, State: Initialized, Role: FOLLOWER
I20260619 12:08:40.891287 18830 ts_tablet_manager.cc:1428] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 12:08:40.891273 18831 consensus_queue.cc:260] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.891480 18813 heartbeater.cc:503] Master 127.0.203.190:39081 was elected leader, sending a full tablet report...
I20260619 12:08:40.891911 18682 heartbeater.cc:503] Master 127.0.203.190:39081 was elected leader, sending a full tablet report...
I20260619 12:08:40.892138 18831 ts_tablet_manager.cc:1428] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 12:08:40.893039 18833 raft_consensus.cc:348] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.893236 18833 raft_consensus.cc:374] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:40.893275 18833 raft_consensus.cc:729] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e7cb6d3e9b498f8ede9e7c93b963ab, State: Initialized, Role: FOLLOWER
I20260619 12:08:40.893455 18833 consensus_queue.cc:260] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.893733 18833 ts_tablet_manager.cc:1428] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 12:08:40.893817 18551 heartbeater.cc:503] Master 127.0.203.190:39081 was elected leader, sending a full tablet report...
I20260619 12:08:40.906430 18836 raft_consensus.cc:482] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:40.906591 18836 raft_consensus.cc:504] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.906996 18836 leader_election.cc:288] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3e7cb6d3e9b498f8ede9e7c93b963ab (127.0.203.129:39431), 347cf481f19a4badaf2b4c61cf82531e (127.0.203.130:43647)
I20260619 12:08:40.911206 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.907298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:48495 (local address 127.0.203.130:43647)
0619 12:08:40.907347 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.907349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.907493 (+ 144us) server_negotiation.cc:407] Connection header received
0619 12:08:40.907559 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.907560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.907573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.907626 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.907769 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.908524 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.910157 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.910760 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.910846 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.910857 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.911061 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.911113 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:08:40.911206 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.907242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:43343 (local address 127.0.203.129:39431)
0619 12:08:40.907295 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.907297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.907316 (+ 19us) server_negotiation.cc:407] Connection header received
0619 12:08:40.907437 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.907439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.907455 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.907519 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:40.907712 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.908522 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.910157 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.910761 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.910847 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.910857 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.911061 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.911115 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:08:40.911233 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.907183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39431 (local address 127.0.203.131:43343)
0619 12:08:40.907274 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.907288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.907334 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.907546 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.907547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.907549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.907630 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.907632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.908647 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.908649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.910021 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.910025 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.910860 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.910861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.910970 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.910979 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.911044 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.911099 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":75}
I20260619 12:08:40.911245 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.907213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:43647 (local address 127.0.203.131:48495)
0619 12:08:40.907421 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.907432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:40.907506 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:40.907645 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:40.907646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:40.907647 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:40.907699 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.907701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.908647 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.908649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.910022 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:40.910025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.910860 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.910861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:40.910970 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.910979 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:08:40.911044 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:08:40.911094 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":62,"thread_start_us":124,"threads_started":1}
I20260619 12:08:40.911872 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecccb352a8bd4cf3a6b3e77dfbdee2f9" candidate_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" is_pre_election: true
I20260619 12:08:40.911870 18636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecccb352a8bd4cf3a6b3e77dfbdee2f9" candidate_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "347cf481f19a4badaf2b4c61cf82531e" is_pre_election: true
I20260619 12:08:40.912060 18505 raft_consensus.cc:2454] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec5ea4d409e4b9a85ebb3379069a85f in term 0.
I20260619 12:08:40.912060 18636 raft_consensus.cc:2454] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec5ea4d409e4b9a85ebb3379069a85f in term 0.
I20260619 12:08:40.912328 18703 leader_election.cc:302] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec5ea4d409e4b9a85ebb3379069a85f, a3e7cb6d3e9b498f8ede9e7c93b963ab; no voters:
I20260619 12:08:40.912498 18836 raft_consensus.cc:2781] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:08:40.912561 18836 raft_consensus.cc:482] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:08:40.912583 18836 raft_consensus.cc:3037] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:40.913363 18836 raft_consensus.cc:504] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.913522 18836 leader_election.cc:288] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [CANDIDATE]: Term 1 election: Requested vote from peers a3e7cb6d3e9b498f8ede9e7c93b963ab (127.0.203.129:39431), 347cf481f19a4badaf2b4c61cf82531e (127.0.203.130:43647)
I20260619 12:08:40.913708 18636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecccb352a8bd4cf3a6b3e77dfbdee2f9" candidate_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "347cf481f19a4badaf2b4c61cf82531e"
I20260619 12:08:40.913686 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecccb352a8bd4cf3a6b3e77dfbdee2f9" candidate_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab"
I20260619 12:08:40.913797 18505 raft_consensus.cc:3037] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:40.913794 18636 raft_consensus.cc:3037] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:40.914708 18636 raft_consensus.cc:2454] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec5ea4d409e4b9a85ebb3379069a85f in term 1.
I20260619 12:08:40.914773 18505 raft_consensus.cc:2454] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec5ea4d409e4b9a85ebb3379069a85f in term 1.
I20260619 12:08:40.914891 18700 leader_election.cc:302] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec5ea4d409e4b9a85ebb3379069a85f, 347cf481f19a4badaf2b4c61cf82531e; no voters:
I20260619 12:08:40.914994 18836 raft_consensus.cc:2781] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:40.915143 18836 raft_consensus.cc:686] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 1 LEADER]: Becoming Leader. State: Replica: 0ec5ea4d409e4b9a85ebb3379069a85f, State: Running, Role: LEADER
I20260619 12:08:40.915235 18836 consensus_queue.cc:237] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } }
I20260619 12:08:40.916225 18363 catalog_manager.cc:5495] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec5ea4d409e4b9a85ebb3379069a85f (127.0.203.131). New cstate: current_term: 1 leader_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec5ea4d409e4b9a85ebb3379069a85f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 44911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 } health_report { overall_health: UNKNOWN } } }
I20260619 12:08:40.930792 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.928386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.0.203.131:44911)
0619 12:08:40.928480 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:40.928483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:40.928492 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:08:40.928539 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:40.928540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:40.928561 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:40.928627 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:40.928770 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.929287 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.930047 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:40.930244 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:40.930299 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:40.930446 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:40.930560 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:08:40.930638 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:40.930655 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 12:08:40.930675 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:08:40.933199 18505 raft_consensus.cc:1261] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 1 FOLLOWER]: Refusing update from remote peer 0ec5ea4d409e4b9a85ebb3379069a85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:40.933200 18636 raft_consensus.cc:1261] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 1 FOLLOWER]: Refusing update from remote peer 0ec5ea4d409e4b9a85ebb3379069a85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:40.933699 18840 consensus_queue.cc:1035] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3e7cb6d3e9b498f8ede9e7c93b963ab" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:40.933907 18836 consensus_queue.cc:1035] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "347cf481f19a4badaf2b4c61cf82531e" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 43647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:40.936836 18843 mvcc.cc:205] Tried to move back new op lower bound from 7298543292138983424 to 7298543292069728256. Current Snapshot: MvccSnapshot[applied={T|T < 7298543292138983424}]
I20260619 12:08:40.937067 18845 mvcc.cc:205] Tried to move back new op lower bound from 7298543292138983424 to 7298543292069728256. Current Snapshot: MvccSnapshot[applied={T|T < 7298543292138983424}]
I20260619 12:08:40.937811 18844 mvcc.cc:205] Tried to move back new op lower bound from 7298543292138983424 to 7298543292069728256. Current Snapshot: MvccSnapshot[applied={T|T < 7298543292138983424}]
I20260619 12:08:40.939873 18362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:40.940008 18362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:40.941051 18362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcd37ea5d4fb4e7f8921efc52826a56e: Sending DeleteTablet for 3 replicas of tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9
I20260619 12:08:40.941428 18747 tablet_service.cc:1514] Processing DeleteTablet for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:40 UTC) from {username='slave'} at 127.0.0.1:60684
I20260619 12:08:40.941473 18485 tablet_service.cc:1514] Processing DeleteTablet for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:40 UTC) from {username='slave'} at 127.0.0.1:44686
I20260619 12:08:40.941609 18616 tablet_service.cc:1514] Processing DeleteTablet for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:40 UTC) from {username='slave'} at 127.0.0.1:55902
I20260619 12:08:40.941890 18853 tablet_replica.cc:330] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab: stopping tablet replica
I20260619 12:08:40.942026 18853 raft_consensus.cc:2229] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:40.942157 18853 raft_consensus.cc:2258] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P a3e7cb6d3e9b498f8ede9e7c93b963ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:40.942155 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18423
I20260619 12:08:40.943353 18852 tablet_replica.cc:330] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: stopping tablet replica
I20260619 12:08:40.943517 18852 raft_consensus.cc:2229] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:08:40.943630 18852 raft_consensus.cc:2258] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:40.944198 18852 ts_tablet_manager.cc:1905] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:40.946306 18852 ts_tablet_manager.cc:1918] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:40.946398 18852 log.cc:1196] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/ecccb352a8bd4cf3a6b3e77dfbdee2f9
I20260619 12:08:40.946849 18852 ts_tablet_manager.cc:1939] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 0ec5ea4d409e4b9a85ebb3379069a85f: Deleting consensus metadata
I20260619 12:08:40.947551 18349 catalog_manager.cc:4841] TS 0ec5ea4d409e4b9a85ebb3379069a85f (127.0.203.131:44911): tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9 (table test-table [id=9d4af1f0661d484fb09f725aa156b8ab]) successfully deleted
I20260619 12:08:40.947925 18854 tablet_replica.cc:330] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: stopping tablet replica
I20260619 12:08:40.948096 18854 raft_consensus.cc:2229] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:40.948251 18854 raft_consensus.cc:2258] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:40.948902 18854 ts_tablet_manager.cc:1905] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:08:40.949731 18350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:39431 (error 108)
I20260619 12:08:40.950322 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.950185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39431 (local address 127.0.0.1:44698)
0619 12:08:40.950239 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.950283 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:39431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 12:08:40.950564 18350 catalog_manager.cc:4584] TS a3e7cb6d3e9b498f8ede9e7c93b963ab (127.0.203.129:39431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ecccb352a8bd4cf3a6b3e77dfbdee2f9: Network error: Client connection negotiation failed: client connection to 127.0.203.129:39431: connect: Connection refused (error 111)
I20260619 12:08:40.950604 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18554
I20260619 12:08:40.950686 18854 ts_tablet_manager.cc:1918] T ecccb352a8bd4cf3a6b3e77dfbdee2f9 P 347cf481f19a4badaf2b4c61cf82531e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:40.955658 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:40.955522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:43647 (local address 127.0.0.1:55916)
0619 12:08:40.955586 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:40.955612 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.130:43647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 12:08:40.956260 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18685
I20260619 12:08:40.961889 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18332
2026-06-19T12:08:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (810 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:41 dist-test-slave-44vj krb5kdc[18858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:41 dist-test-slave-44vj krb5kdc[18858](info): set up 2 sockets
Jun 19 12:08:41 dist-test-slave-44vj krb5kdc[18858](info): commencing operation
krb5kdc: starting...
W20260619 12:08:41.931337 814 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:08:42.063472 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43361
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:43361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:42.152767 18871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:42.153020 18871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:42.153044 18871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:42.154297 18871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:42.154341 18871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:42.154354 18871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:42.154366 18871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:42.154376 18871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:42.155982 18871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:43361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:42.156286 18871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:42.156510 18871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:42.162081 18876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.162156 18877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.162305 18879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:42.162655 18871 server_base.cc:1034] running on GCE node
I20260619 12:08:42.162945 18871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:42.163275 18871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:42.164430 18871 hybrid_clock.cc:648] HybridClock initialized: now 1781870922164408 us; error 46 us; skew 500 ppm
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:42.167423 18871 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:08:42.167954 18871 webserver.cc:460] Webserver started at http://127.0.203.190:33047/ using document root <none> and password file <none>
I20260619 12:08:42.168182 18871 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:42.168228 18871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:42.168344 18871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:42.169485 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "677c4e55a63248cf99b816429d44e195"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "03cfd6ddd183c1d0143cc3d98507a27f"
server_key_iv: "ddf19191d40c15fe239bfbd69ef0cc23"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.169791 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "677c4e55a63248cf99b816429d44e195"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "03cfd6ddd183c1d0143cc3d98507a27f"
server_key_iv: "ddf19191d40c15fe239bfbd69ef0cc23"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.171178 18871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:42.172092 18886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.172288 18871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:42.172399 18871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "677c4e55a63248cf99b816429d44e195"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "03cfd6ddd183c1d0143cc3d98507a27f"
server_key_iv: "ddf19191d40c15fe239bfbd69ef0cc23"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.172485 18871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:42.181303 18871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:42.183748 18871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:42.183909 18871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:42.188135 18871 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:43361
I20260619 12:08:42.188203 18938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:43361 every 8 connection(s)
I20260619 12:08:42.188562 18871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:42.189460 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18871
I20260619 12:08:42.189559 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:42.189639 18939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:42.189746 814 external_mini_cluster.cc:1428] Setting key 29e5fcf7fba9ebfa3e16e9f3af2d8855
I20260619 12:08:42.193128 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195: Bootstrap starting.
I20260619 12:08:42.193864 18939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:42.194206 18939 log.cc:824] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:42.195025 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195: No bootstrap required, opened a new log
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:42.197685 18939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677c4e55a63248cf99b816429d44e195" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43361 } }
I20260619 12:08:42.197919 18939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:42.197957 18939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677c4e55a63248cf99b816429d44e195, State: Initialized, Role: FOLLOWER
I20260619 12:08:42.198114 18939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677c4e55a63248cf99b816429d44e195" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43361 } }
I20260619 12:08:42.198191 18939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:42.198221 18939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:42.198263 18939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:42.199082 18939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677c4e55a63248cf99b816429d44e195" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43361 } }
I20260619 12:08:42.199208 18939 leader_election.cc:302] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 677c4e55a63248cf99b816429d44e195; no voters:
I20260619 12:08:42.199460 18939 leader_election.cc:288] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:42.199559 18944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:42.199805 18939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:42.199803 18944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [term 1 LEADER]: Becoming Leader. State: Replica: 677c4e55a63248cf99b816429d44e195, State: Running, Role: LEADER
I20260619 12:08:42.200011 18944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677c4e55a63248cf99b816429d44e195" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43361 } }
I20260619 12:08:42.200523 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "677c4e55a63248cf99b816429d44e195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677c4e55a63248cf99b816429d44e195" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43361 } } }
I20260619 12:08:42.200668 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:42.201042 18953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:42.201655 18953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:42.202227 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.190313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.0.203.190:43361)
0619 12:08:42.190548 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.190557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.190586 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:08:42.191303 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.191310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.191398 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.191558 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:42.192157 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.193078 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.193971 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.194552 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.194668 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.197650 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:42.197656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:42.197657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:42.197674 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:42.199957 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.200487 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.200491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.200492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.200625 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.200871 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.200872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.200872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.201217 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:42.201390 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.201920 (+ 530us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.202070 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":111,"threads_started":1}
I20260619 12:08:42.202502 18946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 677c4e55a63248cf99b816429d44e195. Latest consensus state: current_term: 1 leader_uuid: "677c4e55a63248cf99b816429d44e195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677c4e55a63248cf99b816429d44e195" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43361 } } }
I20260619 12:08:42.203035 18946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:42.203796 18953 catalog_manager.cc:1347] Generated new cluster ID: 834789c90f1546778cd53f4b5cdbafa4
I20260619 12:08:42.203851 18953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:42.209931 18953 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:42.210528 18953 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:42.218541 18953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195: Generated new TSK 0
I20260619 12:08:42.218866 18953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:08:42.250268 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43361
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:42.338358 18965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:42.338577 18965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:42.338599 18965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:42.339828 18965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:42.339882 18965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:42.339910 18965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:42.341461 18965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:42.341696 18965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:42.341938 18965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:42.347481 18970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.347553 18971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.347522 18973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:42.347543 18965 server_base.cc:1034] running on GCE node
I20260619 12:08:42.348006 18965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:42.348289 18965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:42.349493 18965 hybrid_clock.cc:648] HybridClock initialized: now 1781870922349466 us; error 45 us; skew 500 ppm
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:42.352200 18965 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:08:42.352730 18965 webserver.cc:460] Webserver started at http://127.0.203.129:34733/ using document root <none> and password file <none>
I20260619 12:08:42.352942 18965 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:42.352993 18965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:42.353124 18965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:42.354223 18965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "b93d8f7f61974e448fd52ca60dd71e3f"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "f444c4ad3800fd9e89fdc37e07b06af2"
server_key_iv: "316d32d6614d527c15679aeb8a2b89b2"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.354583 18965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "b93d8f7f61974e448fd52ca60dd71e3f"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "f444c4ad3800fd9e89fdc37e07b06af2"
server_key_iv: "316d32d6614d527c15679aeb8a2b89b2"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.356192 18965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:08:42.357111 18980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.357401 18965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:42.357502 18965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "b93d8f7f61974e448fd52ca60dd71e3f"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "f444c4ad3800fd9e89fdc37e07b06af2"
server_key_iv: "316d32d6614d527c15679aeb8a2b89b2"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.357570 18965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:42.369211 18965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:42.371297 18965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:42.371495 18965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:42.371726 18965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:42.372038 18965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:42.372076 18965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.372105 18965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:42.372123 18965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.378739 18965 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:38289
I20260619 12:08:42.378834 19093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:38289 every 8 connection(s)
I20260619 12:08:42.379149 18965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:08:42.385671 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 18965
I20260619 12:08:42.385780 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:08:42.385962 814 external_mini_cluster.cc:1428] Setting key de6eee87122ad7b4a3d7e9542d9a40d8
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:42.390594 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.380149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:33999 (local address 127.0.203.190:43361)
0619 12:08:42.380203 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.380205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.380363 (+ 158us) server_negotiation.cc:407] Connection header received
0619 12:08:42.381527 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.381529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.381541 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.381624 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:42.382419 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.382962 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.384147 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.384390 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.384468 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.387061 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:42.387063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:42.387063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:42.387072 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:42.388766 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.389396 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.389397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.389397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.389483 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.389790 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.389791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.389791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.389918 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:42.389978 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.390413 (+ 435us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.390519 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:08:42.390856 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.380000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43361 (local address 127.0.203.129:33999)
0619 12:08:42.380197 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.380245 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.381381 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.381711 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.381716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.382038 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:42.382298 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.382302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.383064 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.383064 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.384038 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.384042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.384517 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.384518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.384649 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.385232 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:42.385241 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:42.386921 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:42.388882 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:42.388893 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:42.388902 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:42.389288 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:42.389556 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:42.389557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:42.389558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:42.389715 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:42.389991 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:42.389997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:42.390216 (+ 219us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.390346 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.390559 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260619 12:08:42.391597 19094 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43361
I20260619 12:08:42.391737 19094 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:42.391973 19094 heartbeater.cc:511] Master 127.0.203.190:43361 requested a full tablet report, sending...
I20260619 12:08:42.392763 18903 ts_manager.cc:194] Registered new tserver with Master: b93d8f7f61974e448fd52ca60dd71e3f (127.0.203.129:38289)
I20260619 12:08:42.393620 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:33999
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:08:42.415726 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43361
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:42.499498 19099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:42.499735 19099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:42.499770 19099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:42.501456 19099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:42.501546 19099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:42.501595 19099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:42.503489 19099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:42.503803 19099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:42.504055 19099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:42.510252 19104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.510403 19105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.510255 19107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:42.510582 19099 server_base.cc:1034] running on GCE node
I20260619 12:08:42.510893 19099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:42.511206 19099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:42.512415 19099 hybrid_clock.cc:648] HybridClock initialized: now 1781870922512394 us; error 40 us; skew 500 ppm
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:42.515113 19099 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:08:42.515676 19099 webserver.cc:460] Webserver started at http://127.0.203.130:40331/ using document root <none> and password file <none>
I20260619 12:08:42.515908 19099 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:42.515959 19099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:42.516076 19099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:42.517189 19099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "8b0fb1c219144394a28d9537c2016d67"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "64718e14e53c3105b67ceb39be564574"
server_key_iv: "e1775d7cd10556d3401e9dc420e4bc55"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.517547 19099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "8b0fb1c219144394a28d9537c2016d67"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "64718e14e53c3105b67ceb39be564574"
server_key_iv: "e1775d7cd10556d3401e9dc420e4bc55"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.519037 19099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:42.519958 19114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.520259 19099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:08:42.520385 19099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "8b0fb1c219144394a28d9537c2016d67"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "64718e14e53c3105b67ceb39be564574"
server_key_iv: "e1775d7cd10556d3401e9dc420e4bc55"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.520462 19099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:42.527498 19099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:42.529628 19099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:42.529770 19099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:42.529999 19099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:42.530304 19099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:42.530337 19099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.530366 19099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:42.530395 19099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.537070 19099 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:36771
I20260619 12:08:42.537128 19227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:36771 every 8 connection(s)
I20260619 12:08:42.537457 19099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:42.541657 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 19099
I20260619 12:08:42.541807 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:08:42.542042 814 external_mini_cluster.cc:1428] Setting key 4e5ba43ecf161b2f9c56c113947c6f5e
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:42.548527 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.538407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:43277 (local address 127.0.203.190:43361)
0619 12:08:42.538478 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.538480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.538745 (+ 265us) server_negotiation.cc:407] Connection header received
0619 12:08:42.539777 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.539779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.539794 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.539876 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:42.540643 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.541196 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.542504 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.542817 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.542899 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.545612 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:42.545614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:42.545614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:42.545624 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:42.547013 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.547516 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.547517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.547518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.547576 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.547823 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.547823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.547824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.547953 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:42.548012 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.548317 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.548408 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:08:42.548759 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.538372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43361 (local address 127.0.203.130:43277)
0619 12:08:42.538607 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.538651 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.539654 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.540010 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.540014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.540319 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:42.540540 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.540545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.541325 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.541326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.542365 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.542369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.542920 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.542921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.543055 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.543752 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:42.543763 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:42.545499 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:42.547104 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:42.547116 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:42.547119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:42.547451 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:42.547631 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:42.547631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:42.547632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:42.547761 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:42.548007 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:42.548014 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:42.548183 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.548279 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.548501 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":124,"threads_started":1}
I20260619 12:08:42.549325 19228 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43361
I20260619 12:08:42.549429 19228 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:42.549642 19228 heartbeater.cc:511] Master 127.0.203.190:43361 requested a full tablet report, sending...
I20260619 12:08:42.550127 18903 ts_manager.cc:194] Registered new tserver with Master: 8b0fb1c219144394a28d9537c2016d67 (127.0.203.130:36771)
I20260619 12:08:42.550594 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:43277
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:08:42.570986 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43361
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:42.656831 19233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:42.657097 19233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:42.657135 19233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:42.658464 19233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:42.658530 19233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:42.658574 19233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:42.660362 19233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:43361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:42.660638 19233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:42.660873 19233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:42.666306 19239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.666356 19241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.666313 19238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:42.666456 19233 server_base.cc:1034] running on GCE node
I20260619 12:08:42.666909 19233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:42.667199 19233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:42.668342 19233 hybrid_clock.cc:648] HybridClock initialized: now 1781870922668302 us; error 64 us; skew 500 ppm
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:42.671304 19233 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:08:42.671859 19233 webserver.cc:460] Webserver started at http://127.0.203.131:34277/ using document root <none> and password file <none>
I20260619 12:08:42.672101 19233 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:42.672155 19233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:42.672293 19233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:42.673532 19233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "56041a6c69db4d01bfd2cfb4608cef93"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "75bd6d35a71e8c9528b46ff72db58cd8"
server_key_iv: "af34a335f3718edf2969628de5f2b1f3"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.673909 19233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "56041a6c69db4d01bfd2cfb4608cef93"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "75bd6d35a71e8c9528b46ff72db58cd8"
server_key_iv: "af34a335f3718edf2969628de5f2b1f3"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.675603 19233 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:08:42.676608 19248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.676818 19233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:42.676965 19233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "56041a6c69db4d01bfd2cfb4608cef93"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "75bd6d35a71e8c9528b46ff72db58cd8"
server_key_iv: "af34a335f3718edf2969628de5f2b1f3"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.677063 19233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:42.689276 19233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:42.692008 19233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:42.692165 19233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:42.692407 19233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:42.692756 19233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:42.692793 19233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.692826 19233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:42.692852 19233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.700121 19233 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:46557
I20260619 12:08:42.700333 19361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:46557 every 8 connection(s)
I20260619 12:08:42.700584 19233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:08:42.706872 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 19233
I20260619 12:08:42.706996 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:08:42.707186 814 external_mini_cluster.cc:1428] Setting key 5f97471f8d34a6bf029e45dd079fa6f2
Jun 19 12:08:42 dist-test-slave-44vj krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:42.711530 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.701451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:40659 (local address 127.0.203.190:43361)
0619 12:08:42.701532 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.701535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.701779 (+ 244us) server_negotiation.cc:407] Connection header received
0619 12:08:42.703019 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.703020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.703035 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.703090 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:42.703891 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.704463 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.705526 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.705763 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.705832 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.708536 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:42.708538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:42.708538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:42.708547 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:42.709966 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.710516 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.710517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.710518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.710590 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.710825 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.710826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.710827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.710960 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:42.711018 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.711307 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.711417 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:08:42.711738 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.701452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:43361 (local address 127.0.203.131:40659)
0619 12:08:42.701624 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.701670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.702893 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.703175 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.703181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.703566 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:42.703802 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.703807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.704546 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.704547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.705439 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.705441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.705836 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.705837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.705980 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.706566 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:42.706577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:42.708411 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:42.710103 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:42.710111 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:42.710114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:42.710407 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:42.710651 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:42.710652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:42.710652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:42.710765 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:42.711021 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:42.711028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:42.711180 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.711278 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.711492 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260619 12:08:42.712318 19362 heartbeater.cc:347] Connected to a master server at 127.0.203.190:43361
I20260619 12:08:42.712488 19362 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:42.712888 19362 heartbeater.cc:511] Master 127.0.203.190:43361 requested a full tablet report, sending...
I20260619 12:08:42.713415 18903 ts_manager.cc:194] Registered new tserver with Master: 56041a6c69db4d01bfd2cfb4608cef93 (127.0.203.131:46557)
I20260619 12:08:42.713827 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:40659
I20260619 12:08:42.719520 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:08:42.728421 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.721166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.0.203.190:43361)
0619 12:08:42.721251 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.721253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.721313 (+ 60us) server_negotiation.cc:407] Connection header received
0619 12:08:42.721338 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.721339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.721349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.721401 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:42.722004 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.722517 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.723402 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.723645 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.723721 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.725307 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:42.725309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:42.725309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:42.725317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:42.726705 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.727346 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.727347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.727348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.727492 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:42.727743 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:42.727744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:42.727744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:42.727908 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:42.728006 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.728265 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.728343 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:08:42.730434 18903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:08:42.732332 18903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:08:42.740283 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.736106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.0.203.130:36771)
0619 12:08:42.736486 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.736492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.736503 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:42.736547 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.736550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.736567 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.736678 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:42.736841 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.737612 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.739028 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.739848 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.739924 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.739994 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.740137 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.740182 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":114,"threads_started":1}
I20260619 12:08:42.740276 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.735676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:36771 (local address 127.0.0.1:48958)
0619 12:08:42.736035 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.736047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.736108 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.736680 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.736681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.736685 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:42.736774 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.736777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.737707 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.737709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.738915 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.738918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.739939 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.739940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.740026 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.740041 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.740112 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.740177 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":5,"thread_start_us":57,"threads_started":1}
I20260619 12:08:42.740553 19374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.735676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:46557 (local address 127.0.0.1:34130)
0619 12:08:42.735866 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.735922 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.736030 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.736548 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.736549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.736557 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:42.736689 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.736692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.738257 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.738258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.739243 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.739246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.740264 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.740265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.740347 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.740353 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.740385 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.740446 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260619 12:08:42.741055 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.735817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.0.203.131:46557)
0619 12:08:42.736351 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.736355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.736363 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:08:42.736403 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.736405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.736426 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.736875 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:42.736984 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.738118 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.739335 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.740150 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.740774 (+ 624us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.740859 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.740890 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.740955 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":112,"threads_started":1}
I20260619 12:08:42.741101 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.735892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:38289 (local address 127.0.0.1:35472)
0619 12:08:42.736195 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.736203 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.736248 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.736537 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.736541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.736554 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:42.736782 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.736784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.738235 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.738237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.739447 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.739449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.740858 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.740859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.740943 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.740947 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.740981 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.741036 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":213,"thread_start_us":76,"threads_started":1}
I20260619 12:08:42.741122 19162 tablet_service.cc:1467] Processing CreateTablet for tablet 50b7b694b3724d10a7fec9069ab47bbd (DEFAULT_TABLE table=test-table [id=6d707b08c10b4b4eb0d4a7d43e214304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:42.741484 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.736047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.0.203.129:38289)
0619 12:08:42.736293 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.736300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.736314 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:08:42.736366 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.736369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.736388 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.736605 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:42.736851 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.738091 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.739558 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.740273 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.741267 (+ 994us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.741320 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.741346 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.741404 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
I20260619 12:08:42.741544 19162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50b7b694b3724d10a7fec9069ab47bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:42.742293 19028 tablet_service.cc:1467] Processing CreateTablet for tablet 50b7b694b3724d10a7fec9069ab47bbd (DEFAULT_TABLE table=test-table [id=6d707b08c10b4b4eb0d4a7d43e214304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:42.742657 19028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50b7b694b3724d10a7fec9069ab47bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:42.744457 19296 tablet_service.cc:1467] Processing CreateTablet for tablet 50b7b694b3724d10a7fec9069ab47bbd (DEFAULT_TABLE table=test-table [id=6d707b08c10b4b4eb0d4a7d43e214304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:42.744868 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50b7b694b3724d10a7fec9069ab47bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:42.745428 19379 tablet_bootstrap.cc:492] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: Bootstrap starting.
I20260619 12:08:42.746212 19379 tablet_bootstrap.cc:654] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:42.746232 19380 tablet_bootstrap.cc:492] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f: Bootstrap starting.
I20260619 12:08:42.746847 19380 tablet_bootstrap.cc:654] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:42.746824 19379 log.cc:824] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:42.747483 19380 log.cc:824] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:42.747928 19379 tablet_bootstrap.cc:492] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: No bootstrap required, opened a new log
I20260619 12:08:42.748026 19379 ts_tablet_manager.cc:1397] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:08:42.748404 19380 tablet_bootstrap.cc:492] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f: No bootstrap required, opened a new log
I20260619 12:08:42.748523 19380 ts_tablet_manager.cc:1397] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:08:42.748996 19383 tablet_bootstrap.cc:492] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Bootstrap starting.
I20260619 12:08:42.749810 19383 tablet_bootstrap.cc:654] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:42.750195 19379 raft_consensus.cc:348] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.750422 19379 raft_consensus.cc:374] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:42.750463 19379 raft_consensus.cc:729] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0fb1c219144394a28d9537c2016d67, State: Initialized, Role: FOLLOWER
I20260619 12:08:42.750339 19380 raft_consensus.cc:348] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.750514 19380 raft_consensus.cc:374] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:42.750541 19380 raft_consensus.cc:729] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b93d8f7f61974e448fd52ca60dd71e3f, State: Initialized, Role: FOLLOWER
I20260619 12:08:42.750597 19379 consensus_queue.cc:260] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.750677 19380 consensus_queue.cc:260] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.750850 19383 log.cc:824] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:42.750921 19379 ts_tablet_manager.cc:1428] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:08:42.751000 19228 heartbeater.cc:503] Master 127.0.203.190:43361 was elected leader, sending a full tablet report...
I20260619 12:08:42.751032 19380 ts_tablet_manager.cc:1428] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 12:08:42.751152 19094 heartbeater.cc:503] Master 127.0.203.190:43361 was elected leader, sending a full tablet report...
I20260619 12:08:42.751720 19383 tablet_bootstrap.cc:492] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: No bootstrap required, opened a new log
I20260619 12:08:42.751794 19383 ts_tablet_manager.cc:1397] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:08:42.753578 19383 raft_consensus.cc:348] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.753734 19383 raft_consensus.cc:374] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:42.753770 19383 raft_consensus.cc:729] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56041a6c69db4d01bfd2cfb4608cef93, State: Initialized, Role: FOLLOWER
I20260619 12:08:42.753903 19383 consensus_queue.cc:260] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.754176 19383 ts_tablet_manager.cc:1428] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:08:42.754241 19362 heartbeater.cc:503] Master 127.0.203.190:43361 was elected leader, sending a full tablet report...
I20260619 12:08:42.758332 19387 raft_consensus.cc:482] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:42.758471 19387 raft_consensus.cc:504] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.758864 19387 leader_election.cc:288] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b0fb1c219144394a28d9537c2016d67 (127.0.203.130:36771), b93d8f7f61974e448fd52ca60dd71e3f (127.0.203.129:38289)
I20260619 12:08:42.762126 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.759170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:43961 (local address 127.0.203.130:36771)
0619 12:08:42.759436 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.759438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.759446 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:08:42.759471 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.759471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.759479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.759532 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:42.759684 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.760239 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.761339 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.761794 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.761858 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.761869 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.762011 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.762050 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260619 12:08:42.762128 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.759055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:36771 (local address 127.0.203.131:43961)
0619 12:08:42.759238 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.759252 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.759323 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.759545 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.759546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.759548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:42.759628 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.759630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.760315 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.760317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.761256 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.761258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.761862 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.761863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.761949 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.761952 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.761999 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.762049 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260619 12:08:42.762403 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.759330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:48605 (local address 127.0.203.129:38289)
0619 12:08:42.759411 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.759413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.759534 (+ 121us) server_negotiation.cc:407] Connection header received
0619 12:08:42.759607 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.759608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.759618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.759668 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:42.759826 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.760351 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.761563 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.761980 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.762050 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.762060 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.762257 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.762321 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:08:42.762501 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b7b694b3724d10a7fec9069ab47bbd" candidate_uuid: "56041a6c69db4d01bfd2cfb4608cef93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0fb1c219144394a28d9537c2016d67" is_pre_election: true
I20260619 12:08:42.762677 19182 raft_consensus.cc:2454] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56041a6c69db4d01bfd2cfb4608cef93 in term 0.
I20260619 12:08:42.762753 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.759173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:38289 (local address 127.0.203.131:48605)
0619 12:08:42.759443 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.759473 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:42.759541 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:42.759675 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:42.759675 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:42.759677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:42.759769 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.759771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.760447 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.760448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.761418 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:42.761421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.762105 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.762106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:42.762167 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.762171 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:42.762597 (+ 426us) client_negotiation.cc:240] Negotiation successful
0619 12:08:42.762658 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":98,"threads_started":1}
I20260619 12:08:42.762880 19249 leader_election.cc:302] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56041a6c69db4d01bfd2cfb4608cef93, 8b0fb1c219144394a28d9537c2016d67; no voters:
I20260619 12:08:42.763002 19387 raft_consensus.cc:2781] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:08:42.763064 19387 raft_consensus.cc:482] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:08:42.763088 19387 raft_consensus.cc:3037] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:42.763301 19048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b7b694b3724d10a7fec9069ab47bbd" candidate_uuid: "56041a6c69db4d01bfd2cfb4608cef93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" is_pre_election: true
I20260619 12:08:42.763511 19048 raft_consensus.cc:2454] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56041a6c69db4d01bfd2cfb4608cef93 in term 0.
I20260619 12:08:42.764055 19387 raft_consensus.cc:504] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.764189 19387 leader_election.cc:288] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [CANDIDATE]: Term 1 election: Requested vote from peers 8b0fb1c219144394a28d9537c2016d67 (127.0.203.130:36771), b93d8f7f61974e448fd52ca60dd71e3f (127.0.203.129:38289)
I20260619 12:08:42.764365 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b7b694b3724d10a7fec9069ab47bbd" candidate_uuid: "56041a6c69db4d01bfd2cfb4608cef93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0fb1c219144394a28d9537c2016d67"
I20260619 12:08:42.764387 19048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50b7b694b3724d10a7fec9069ab47bbd" candidate_uuid: "56041a6c69db4d01bfd2cfb4608cef93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b93d8f7f61974e448fd52ca60dd71e3f"
I20260619 12:08:42.764487 19048 raft_consensus.cc:3037] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:42.764446 19182 raft_consensus.cc:3037] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:42.765283 19182 raft_consensus.cc:2454] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56041a6c69db4d01bfd2cfb4608cef93 in term 1.
I20260619 12:08:42.765415 19048 raft_consensus.cc:2454] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56041a6c69db4d01bfd2cfb4608cef93 in term 1.
I20260619 12:08:42.765450 19249 leader_election.cc:302] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56041a6c69db4d01bfd2cfb4608cef93, 8b0fb1c219144394a28d9537c2016d67; no voters:
I20260619 12:08:42.765594 19387 raft_consensus.cc:2781] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:42.765758 19387 raft_consensus.cc:686] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 1 LEADER]: Becoming Leader. State: Replica: 56041a6c69db4d01bfd2cfb4608cef93, State: Running, Role: LEADER
I20260619 12:08:42.765851 19387 consensus_queue.cc:237] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } }
I20260619 12:08:42.766824 18902 catalog_manager.cc:5495] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56041a6c69db4d01bfd2cfb4608cef93 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "56041a6c69db4d01bfd2cfb4608cef93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56041a6c69db4d01bfd2cfb4608cef93" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 46557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 } health_report { overall_health: UNKNOWN } } }
I20260619 12:08:42.779294 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.775708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.0.203.131:46557)
0619 12:08:42.775834 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.775837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.775851 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:08:42.775888 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.775889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.775907 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.775987 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:42.776251 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.777019 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.778218 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.778434 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.778504 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.778855 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:42.778963 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:08:42.779062 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.779172 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.779193 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 12:08:42.781919 19048 raft_consensus.cc:1261] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 1 FOLLOWER]: Refusing update from remote peer 56041a6c69db4d01bfd2cfb4608cef93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:42.781971 19182 raft_consensus.cc:1261] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 1 FOLLOWER]: Refusing update from remote peer 56041a6c69db4d01bfd2cfb4608cef93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:42.782305 19387 consensus_queue.cc:1035] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b93d8f7f61974e448fd52ca60dd71e3f" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 38289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:42.782473 19389 consensus_queue.cc:1035] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b0fb1c219144394a28d9537c2016d67" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 36771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:42.786353 19393 mvcc.cc:205] Tried to move back new op lower bound from 7298543299710988288 to 7298543299649622016. Current Snapshot: MvccSnapshot[applied={T|T < 7298543299710988288}]
I20260619 12:08:42.788872 18902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:42.789027 18902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:42.790061 18902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 677c4e55a63248cf99b816429d44e195: Sending DeleteTablet for 3 replicas of tablet 50b7b694b3724d10a7fec9069ab47bbd
I20260619 12:08:42.790505 19162 tablet_service.cc:1514] Processing DeleteTablet for tablet 50b7b694b3724d10a7fec9069ab47bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:42 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:48958
I20260619 12:08:42.790575 19296 tablet_service.cc:1514] Processing DeleteTablet for tablet 50b7b694b3724d10a7fec9069ab47bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:42 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:34130
I20260619 12:08:42.790549 19028 tablet_service.cc:1514] Processing DeleteTablet for tablet 50b7b694b3724d10a7fec9069ab47bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:42 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:35472
I20260619 12:08:42.790899 19403 tablet_replica.cc:330] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f: stopping tablet replica
I20260619 12:08:42.791061 19403 raft_consensus.cc:2229] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:42.791203 19403 raft_consensus.cc:2258] T 50b7b694b3724d10a7fec9069ab47bbd P b93d8f7f61974e448fd52ca60dd71e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:42.791185 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18965
I20260619 12:08:42.795562 19401 tablet_replica.cc:330] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: stopping tablet replica
I20260619 12:08:42.795784 19401 raft_consensus.cc:2229] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:42.795944 19401 raft_consensus.cc:2258] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:42.796180 19402 tablet_replica.cc:330] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: stopping tablet replica
I20260619 12:08:42.796314 19402 raft_consensus.cc:2229] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:08:42.796473 19402 raft_consensus.cc:2258] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:42.796768 19401 ts_tablet_manager.cc:1905] T 50b7b694b3724d10a7fec9069ab47bbd P 8b0fb1c219144394a28d9537c2016d67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:42.797055 19402 ts_tablet_manager.cc:1905] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:08:42.797616 18887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:38289 (error 108)
I20260619 12:08:42.798413 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 19099
I20260619 12:08:42.799189 19402 ts_tablet_manager.cc:1918] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:42.799309 19402 log.cc:1196] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/50b7b694b3724d10a7fec9069ab47bbd
I20260619 12:08:42.799623 19402 ts_tablet_manager.cc:1939] T 50b7b694b3724d10a7fec9069ab47bbd P 56041a6c69db4d01bfd2cfb4608cef93: Deleting consensus metadata
I20260619 12:08:42.800204 18888 catalog_manager.cc:4841] TS 56041a6c69db4d01bfd2cfb4608cef93 (127.0.203.131:46557): tablet 50b7b694b3724d10a7fec9069ab47bbd (table test-table [id=6d707b08c10b4b4eb0d4a7d43e214304]) successfully deleted
I20260619 12:08:42.801401 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.801160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:38289 (local address 127.0.0.1:35484)
0619 12:08:42.801254 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.801349 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:38289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 12:08:42.801862 18887 catalog_manager.cc:4584] TS b93d8f7f61974e448fd52ca60dd71e3f (127.0.203.129:38289): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50b7b694b3724d10a7fec9069ab47bbd: Network error: Client connection negotiation failed: client connection to 127.0.203.129:38289: connect: Connection refused (error 111)
I20260619 12:08:42.804930 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 19233
I20260619 12:08:42.805080 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.804888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:36771 (local address 127.0.0.1:48968)
0619 12:08:42.804960 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:42.805033 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.130:36771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 12:08:42.810647 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 18871
2026-06-19T12:08:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1850 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T12:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:42Z Disabled control of system clock
I20260619 12:08:42.840993 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33873
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:40999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:42.928443 19408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:42.928730 19408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:42.928751 19408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:42.930043 19408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:42.930096 19408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:42.930107 19408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:42.930119 19408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:42.930130 19408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:42.931592 19408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33873
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:42.931869 19408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:42.932245 19408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:42.937548 19416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.937595 19413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:42.937561 19414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:42.938269 19408 server_base.cc:1034] running on GCE node
I20260619 12:08:42.938552 19408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:42.938890 19408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:42.940047 19408 hybrid_clock.cc:648] HybridClock initialized: now 1781870922940031 us; error 35 us; skew 500 ppm
I20260619 12:08:42.941464 19408 webserver.cc:460] Webserver started at http://127.0.203.190:42069/ using document root <none> and password file <none>
I20260619 12:08:42.941727 19408 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:42.941782 19408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:42.941916 19408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:42.943097 19408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "6a67cf9519b94c3183caccb45d4c6a60"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "7151db01c8d2478ea60b1eab24187984"
server_key_iv: "d5e2baf13b394a1da0ded58a8bdf520d"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.943486 19408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "6a67cf9519b94c3183caccb45d4c6a60"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "7151db01c8d2478ea60b1eab24187984"
server_key_iv: "d5e2baf13b394a1da0ded58a8bdf520d"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.944967 19408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:42.945932 19422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:42.946188 19408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:42.946281 19408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "6a67cf9519b94c3183caccb45d4c6a60"
format_stamp: "Formatted at 2026-06-19 12:08:42 on dist-test-slave-44vj"
server_key: "7151db01c8d2478ea60b1eab24187984"
server_key_iv: "d5e2baf13b394a1da0ded58a8bdf520d"
server_key_version: "encryptionkey@0"
I20260619 12:08:42.946347 19408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:42.961669 19408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:42.962064 19408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:42.962208 19408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:42.967234 19408 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:33873
I20260619 12:08:42.967346 19474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:33873 every 8 connection(s)
I20260619 12:08:42.967710 19408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:42.968773 19475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:42.971674 19475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60: Bootstrap starting.
I20260619 12:08:42.972294 19475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:42.972558 19475 log.cc:824] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:42.973356 19475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60: No bootstrap required, opened a new log
I20260619 12:08:42.975207 19475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a67cf9519b94c3183caccb45d4c6a60" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33873 } }
I20260619 12:08:42.975450 19475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:42.975486 19475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a67cf9519b94c3183caccb45d4c6a60, State: Initialized, Role: FOLLOWER
I20260619 12:08:42.975615 19475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a67cf9519b94c3183caccb45d4c6a60" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33873 } }
I20260619 12:08:42.975683 19475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:42.975705 19475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:42.975745 19475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:42.976346 19475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a67cf9519b94c3183caccb45d4c6a60" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33873 } }
I20260619 12:08:42.976449 19475 leader_election.cc:302] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a67cf9519b94c3183caccb45d4c6a60; no voters:
I20260619 12:08:42.976608 19475 leader_election.cc:288] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:42.976701 19478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:42.976876 19475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:42.977036 19478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [term 1 LEADER]: Becoming Leader. State: Replica: 6a67cf9519b94c3183caccb45d4c6a60, State: Running, Role: LEADER
I20260619 12:08:42.977190 19478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a67cf9519b94c3183caccb45d4c6a60" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33873 } }
I20260619 12:08:42.977406 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 19408
I20260619 12:08:42.977525 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:42.977754 814 external_mini_cluster.cc:1428] Setting key 5b7bf12be2f86da48c2134810e3253ae
I20260619 12:08:42.977726 19480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a67cf9519b94c3183caccb45d4c6a60. Latest consensus state: current_term: 1 leader_uuid: "6a67cf9519b94c3183caccb45d4c6a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a67cf9519b94c3183caccb45d4c6a60" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33873 } } }
I20260619 12:08:42.977806 19480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:42.977947 19479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a67cf9519b94c3183caccb45d4c6a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a67cf9519b94c3183caccb45d4c6a60" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33873 } } }
I20260619 12:08:42.978005 19479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:42.978062 19487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:42.978693 19487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:42.980626 19487 catalog_manager.cc:1347] Generated new cluster ID: b7ed092f8a9849868c4ec7d7ccd06d89
I20260619 12:08:42.980685 19487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:42.984669 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:42.978299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.0.203.190:33873)
0619 12:08:42.978628 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:42.978636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:42.978662 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:08:42.978787 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:42.978796 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:42.978876 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:42.979017 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:42.979251 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.980753 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.981855 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:42.982620 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:42.983173 (+ 553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:42.983673 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:42.983677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:42.983678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:42.983900 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:42.983944 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:42.983960 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:42.984288 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:42.984323 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:08:42.984415 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":116,"threads_started":1}
I20260619 12:08:42.988225 19487 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:42.988853 19487 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:42.999884 19487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60: Generated new TSK 0
I20260619 12:08:43.000241 19487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:43.002220 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33873
--builtin_ntp_servers=127.0.203.148:40999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:43.091814 19499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:43.092072 19499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:43.092108 19499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:43.093452 19499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:43.093518 19499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:43.093560 19499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:43.095423 19499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:43.095734 19499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:43.096102 19499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:43.101387 19504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:43.101488 19505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:43.101354 19507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:43.101869 19499 server_base.cc:1034] running on GCE node
I20260619 12:08:43.102133 19499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:43.102459 19499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:43.103622 19499 hybrid_clock.cc:648] HybridClock initialized: now 1781870923103611 us; error 31 us; skew 500 ppm
I20260619 12:08:43.105417 19499 webserver.cc:460] Webserver started at http://127.0.203.129:34887/ using document root <none> and password file <none>
I20260619 12:08:43.105741 19499 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:43.105808 19499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:43.105955 19499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:43.107599 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "c5d514fd628e4a21b0a350caef89814a"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "7bfa7dbf9430aefe43ac945a498d3612"
server_key_iv: "c671a81ebc6abfd9137c42d2959c0a5f"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.108067 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "c5d514fd628e4a21b0a350caef89814a"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "7bfa7dbf9430aefe43ac945a498d3612"
server_key_iv: "c671a81ebc6abfd9137c42d2959c0a5f"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.110018 19499 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:43.111195 19513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.111495 19499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:43.111630 19499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "c5d514fd628e4a21b0a350caef89814a"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "7bfa7dbf9430aefe43ac945a498d3612"
server_key_iv: "c671a81ebc6abfd9137c42d2959c0a5f"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.111721 19499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:43.133020 19499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:43.133387 19499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:43.133504 19499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:43.133766 19499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:43.134161 19499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:43.134196 19499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.134228 19499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:43.134244 19499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.141254 19499 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:43115
I20260619 12:08:43.141317 19626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:43115 every 8 connection(s)
I20260619 12:08:43.141696 19499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:08:43.147784 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.142695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:46985 (local address 127.0.203.190:33873)
0619 12:08:43.142798 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.142800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.143025 (+ 225us) server_negotiation.cc:407] Connection header received
0619 12:08:43.143675 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.143676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.143694 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.143792 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:43.144581 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.145124 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.146247 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.146484 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.146575 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.147486 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:43.147488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:43.147488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:43.147494 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:43.147515 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:43.147519 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:43.147581 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.147674 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.147714 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 12:08:43.147989 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.142657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33873 (local address 127.0.203.129:46985)
0619 12:08:43.142878 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.142926 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.143540 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.143885 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.143889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.143899 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:43.144472 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.144476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.145244 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.145245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.146127 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.146130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.146592 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.146592 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.146717 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.147278 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:43.147278 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:43.147310 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:43.147335 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:43.147593 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:43.147598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:43.147601 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.147668 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.147766 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260619 12:08:43.148658 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 19499
I20260619 12:08:43.148772 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:08:43.148758 19627 heartbeater.cc:347] Connected to a master server at 127.0.203.190:33873
I20260619 12:08:43.148913 19627 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:43.148986 814 external_mini_cluster.cc:1428] Setting key 51d05795be1a84d46986be7063a71c38
I20260619 12:08:43.149222 19627 heartbeater.cc:511] Master 127.0.203.190:33873 requested a full tablet report, sending...
I20260619 12:08:43.149957 19439 ts_manager.cc:194] Registered new tserver with Master: c5d514fd628e4a21b0a350caef89814a (127.0.203.129:43115)
I20260619 12:08:43.150177 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33873
--builtin_ntp_servers=127.0.203.148:40999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:08:43.150889 19439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.129:46985
W20260619 12:08:43.235268 19630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:43.235558 19630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:43.235585 19630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:43.236820 19630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:43.236864 19630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:43.236896 19630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:43.238294 19630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:43.238513 19630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:43.238791 19630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:08:43.243979 19630 server_base.cc:1034] running on GCE node
W20260619 12:08:43.243991 19636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:43.244053 19635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:43.244151 19638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:43.244503 19630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:43.244742 19630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:43.245895 19630 hybrid_clock.cc:648] HybridClock initialized: now 1781870923245871 us; error 45 us; skew 500 ppm
I20260619 12:08:43.250056 19630 webserver.cc:460] Webserver started at http://127.0.203.130:43201/ using document root <none> and password file <none>
I20260619 12:08:43.250319 19630 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:43.250370 19630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:43.250530 19630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:43.251701 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "bc01445340b04985b3b591117da62309"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "315133945d1d6029c4a050d5619b11a0"
server_key_iv: "98c8b2af9850765e53e936293cb5c8ab"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.252030 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "bc01445340b04985b3b591117da62309"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "315133945d1d6029c4a050d5619b11a0"
server_key_iv: "98c8b2af9850765e53e936293cb5c8ab"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.253376 19630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:08:43.254261 19644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.254487 19630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:43.254576 19630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "bc01445340b04985b3b591117da62309"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "315133945d1d6029c4a050d5619b11a0"
server_key_iv: "98c8b2af9850765e53e936293cb5c8ab"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.254645 19630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:43.267524 19630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:43.267845 19630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:43.267977 19630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:43.268187 19630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:43.268532 19630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:43.268572 19630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.268602 19630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:43.268618 19630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.275849 19630 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:40701
I20260619 12:08:43.275904 19757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:40701 every 8 connection(s)
I20260619 12:08:43.276234 19630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:43.281881 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.277117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:36937 (local address 127.0.203.190:33873)
0619 12:08:43.277181 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.277183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.277410 (+ 227us) server_negotiation.cc:407] Connection header received
0619 12:08:43.277911 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.277913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.277925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.278017 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:43.278758 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.279393 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.280396 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.280612 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.280674 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.281554 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:43.281556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:43.281556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:43.281561 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:43.281581 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:43.281585 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:43.281642 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.281758 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.281792 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:08:43.281984 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.277059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33873 (local address 127.0.203.130:36937)
0619 12:08:43.277253 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.277305 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.277785 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.278095 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.278100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.278110 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:43.278644 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.278649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.279482 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.279483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.280294 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.280297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.280688 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.280689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.280800 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.281358 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:43.281359 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:43.281390 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:43.281434 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:43.281652 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:43.281655 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:43.281657 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.281713 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.281803 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260619 12:08:43.282514 19758 heartbeater.cc:347] Connected to a master server at 127.0.203.190:33873
I20260619 12:08:43.282642 19758 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:43.282876 19758 heartbeater.cc:511] Master 127.0.203.190:33873 requested a full tablet report, sending...
I20260619 12:08:43.283439 19439 ts_manager.cc:194] Registered new tserver with Master: bc01445340b04985b3b591117da62309 (127.0.203.130:40701)
I20260619 12:08:43.283838 19439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.130:36937
I20260619 12:08:43.285745 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 19630
I20260619 12:08:43.285848 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:08:43.286051 814 external_mini_cluster.cc:1428] Setting key 1b7b19be77374a03ee8a7aff4bb13b8a
I20260619 12:08:43.287149 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33873
--builtin_ntp_servers=127.0.203.148:40999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:43.371419 19761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:43.371649 19761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:43.371675 19761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:43.372979 19761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:43.373034 19761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:43.373061 19761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:43.374516 19761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:33873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:43.374828 19761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:43.375234 19761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:43.381695 19766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:43.381815 19769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:43.381695 19767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:43.382251 19761 server_base.cc:1034] running on GCE node
I20260619 12:08:43.382488 19761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:43.382722 19761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:43.383859 19761 hybrid_clock.cc:648] HybridClock initialized: now 1781870923383844 us; error 37 us; skew 500 ppm
I20260619 12:08:43.385125 19761 webserver.cc:460] Webserver started at http://127.0.203.131:36365/ using document root <none> and password file <none>
I20260619 12:08:43.385353 19761 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:43.385414 19761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:43.385507 19761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:43.386538 19761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "c5e4dfeb11924f4096897a1efc016e4f"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "a27b5b410defb10acc7c9557b4597c57"
server_key_iv: "69360693ba0d7da64c7369987417f46e"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.386821 19761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "c5e4dfeb11924f4096897a1efc016e4f"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "a27b5b410defb10acc7c9557b4597c57"
server_key_iv: "69360693ba0d7da64c7369987417f46e"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.388128 19761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:43.388892 19775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.389130 19761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:43.389215 19761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "c5e4dfeb11924f4096897a1efc016e4f"
format_stamp: "Formatted at 2026-06-19 12:08:43 on dist-test-slave-44vj"
server_key: "a27b5b410defb10acc7c9557b4597c57"
server_key_iv: "69360693ba0d7da64c7369987417f46e"
server_key_version: "encryptionkey@0"
I20260619 12:08:43.389302 19761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:43.400142 19761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:43.400471 19761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:43.400600 19761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:43.400822 19761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:43.401151 19761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:43.401185 19761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.401216 19761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:43.401232 19761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:43.407765 19761 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:42639
I20260619 12:08:43.407817 19888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:42639 every 8 connection(s)
I20260619 12:08:43.408154 19761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:08:43.412787 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 19761
I20260619 12:08:43.412931 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:08:43.413143 814 external_mini_cluster.cc:1428] Setting key 8851716b27c59b20e656bf7d9e73567d
I20260619 12:08:43.414563 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.409039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:56787 (local address 127.0.203.190:33873)
0619 12:08:43.409130 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.409133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.409509 (+ 376us) server_negotiation.cc:407] Connection header received
0619 12:08:43.410057 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.410059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.410073 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.410144 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:43.410888 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.411520 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.412621 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.412926 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.413012 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.414203 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:43.414205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:43.414205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:43.414210 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:43.414228 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:43.414232 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:43.414287 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.414452 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.414486 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:08:43.414790 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.409040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:33873 (local address 127.0.203.131:56787)
0619 12:08:43.409316 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.409364 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.409920 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.410220 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.410224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.410235 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:43.410778 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.410794 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.411645 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.411646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.412520 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.412523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.413115 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.413116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.413299 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.414042 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:43.414042 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:43.414085 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:43.414120 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:43.414306 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:43.414315 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:43.414319 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.414420 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.414534 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":124,"threads_started":1}
I20260619 12:08:43.415341 19889 heartbeater.cc:347] Connected to a master server at 127.0.203.190:33873
I20260619 12:08:43.415488 19889 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:43.415711 19889 heartbeater.cc:511] Master 127.0.203.190:33873 requested a full tablet report, sending...
I20260619 12:08:43.416162 19439 ts_manager.cc:194] Registered new tserver with Master: c5e4dfeb11924f4096897a1efc016e4f (127.0.203.131:42639)
I20260619 12:08:43.416529 19439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.131:56787
I20260619 12:08:43.425712 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:08:43.430527 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.427310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.0.203.190:33873)
0619 12:08:43.427411 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.427413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.427489 (+ 76us) server_negotiation.cc:407] Connection header received
0619 12:08:43.427565 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.427566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.427581 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.427630 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:43.427813 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.428379 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.429175 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.429357 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.429413 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.430233 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:43.430235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:43.430235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:43.430240 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:43.430255 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:43.430260 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:43.430337 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.430423 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.430435 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 12:08:43.432790 19439 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:08:43.434571 19439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:08:43.442292 19902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.437653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.0.203.129:43115)
0619 12:08:43.437823 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.437828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.437965 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:08:43.438015 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.438017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.438034 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.438117 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:43.438328 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.439583 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.441057 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.441817 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.441893 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.441973 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.442127 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.442175 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260619 12:08:43.442319 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.437148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:42639 (local address 127.0.0.1:59466)
0619 12:08:43.437449 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.437485 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.437551 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.438418 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.438419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.438422 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:43.438492 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.438496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.439677 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.439678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.441136 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.441139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.442057 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.442058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.442122 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.442136 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.442184 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.442230 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":76,"thread_start_us":177,"threads_started":1}
I20260619 12:08:43.442302 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.437147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:43115 (local address 127.0.0.1:60932)
0619 12:08:43.437842 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.437853 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.437913 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.438191 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.438191 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.438195 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:43.438233 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.438235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.439655 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.439656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.440931 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.440935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.441913 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.441914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.441997 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.442015 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.442094 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.442179 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":14,"thread_start_us":115,"threads_started":1}
I20260619 12:08:43.442715 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.437265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.0.203.131:42639)
0619 12:08:43.438266 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.438270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.438280 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:08:43.438316 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.438317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.438332 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.438402 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:43.438661 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.439559 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.441228 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.441830 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.441897 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.442019 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.442561 (+ 542us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.442626 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":110,"threads_started":1}
I20260619 12:08:43.442934 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.437432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.0.203.130:40701)
0619 12:08:43.437683 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.437687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.437697 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:08:43.437735 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.437736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.437750 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.437825 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:43.438084 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.439290 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.440902 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.441715 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.441799 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.441919 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.442702 (+ 783us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.442759 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":126,"threads_started":1}
I20260619 12:08:43.443089 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.437147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:40701 (local address 127.0.0.1:40540)
0619 12:08:43.437423 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.437469 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.437541 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.437895 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.437898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.437902 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:43.437998 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.438002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.439493 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.439495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.440786 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.440790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.441967 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.441968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.442055 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.442063 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.442114 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.442184 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":66,"threads_started":1}
I20260619 12:08:43.443612 19823 tablet_service.cc:1467] Processing CreateTablet for tablet 7b0af12994264e02959417f464c8e583 (DEFAULT_TABLE table=test-table [id=197016a27cec43e5a2f3f2de47ed8694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:43.443956 19692 tablet_service.cc:1467] Processing CreateTablet for tablet 7b0af12994264e02959417f464c8e583 (DEFAULT_TABLE table=test-table [id=197016a27cec43e5a2f3f2de47ed8694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:43.444082 19823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b0af12994264e02959417f464c8e583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:43.444011 19561 tablet_service.cc:1467] Processing CreateTablet for tablet 7b0af12994264e02959417f464c8e583 (DEFAULT_TABLE table=test-table [id=197016a27cec43e5a2f3f2de47ed8694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:43.444291 19692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b0af12994264e02959417f464c8e583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:43.444404 19561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b0af12994264e02959417f464c8e583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:43.447443 19903 tablet_bootstrap.cc:492] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a: Bootstrap starting.
I20260619 12:08:43.448504 19903 tablet_bootstrap.cc:654] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:43.448465 19905 tablet_bootstrap.cc:492] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Bootstrap starting.
I20260619 12:08:43.448825 19903 log.cc:824] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:43.449286 19905 tablet_bootstrap.cc:654] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:43.449640 19905 log.cc:824] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:43.450038 19904 tablet_bootstrap.cc:492] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Bootstrap starting.
I20260619 12:08:43.450543 19905 tablet_bootstrap.cc:492] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: No bootstrap required, opened a new log
I20260619 12:08:43.450606 19905 ts_tablet_manager.cc:1397] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:08:43.450788 19904 tablet_bootstrap.cc:654] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:43.451114 19904 log.cc:824] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:43.451321 19903 tablet_bootstrap.cc:492] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a: No bootstrap required, opened a new log
I20260619 12:08:43.451517 19903 ts_tablet_manager.cc:1397] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260619 12:08:43.451916 19904 tablet_bootstrap.cc:492] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: No bootstrap required, opened a new log
I20260619 12:08:43.451987 19904 ts_tablet_manager.cc:1397] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:08:43.453305 19905 raft_consensus.cc:348] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.453730 19905 raft_consensus.cc:374] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:43.453778 19905 raft_consensus.cc:729] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc01445340b04985b3b591117da62309, State: Initialized, Role: FOLLOWER
I20260619 12:08:43.453925 19905 consensus_queue.cc:260] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.453974 19903 raft_consensus.cc:348] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.454170 19903 raft_consensus.cc:374] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:43.454205 19903 raft_consensus.cc:729] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d514fd628e4a21b0a350caef89814a, State: Initialized, Role: FOLLOWER
I20260619 12:08:43.454212 19905 ts_tablet_manager.cc:1428] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 12:08:43.454339 19758 heartbeater.cc:503] Master 127.0.203.190:33873 was elected leader, sending a full tablet report...
I20260619 12:08:43.454351 19903 consensus_queue.cc:260] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.454751 19627 heartbeater.cc:503] Master 127.0.203.190:33873 was elected leader, sending a full tablet report...
I20260619 12:08:43.454847 19903 ts_tablet_manager.cc:1428] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 12:08:43.454542 19904 raft_consensus.cc:348] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.455049 19904 raft_consensus.cc:374] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:43.455101 19904 raft_consensus.cc:729] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e4dfeb11924f4096897a1efc016e4f, State: Initialized, Role: FOLLOWER
I20260619 12:08:43.455225 19904 consensus_queue.cc:260] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.455549 19904 ts_tablet_manager.cc:1428] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260619 12:08:43.455766 19889 heartbeater.cc:503] Master 127.0.203.190:33873 was elected leader, sending a full tablet report...
W20260619 12:08:43.526870 19759 tablet.cc:2369] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:43.591892 19911 raft_consensus.cc:482] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:43.592031 19911 raft_consensus.cc:504] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.592460 19911 leader_election.cc:288] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5d514fd628e4a21b0a350caef89814a (127.0.203.129:43115), bc01445340b04985b3b591117da62309 (127.0.203.130:40701)
I20260619 12:08:43.596848 19902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.592917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:40731 (local address 127.0.203.129:43115)
0619 12:08:43.592963 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.592965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.593090 (+ 125us) server_negotiation.cc:407] Connection header received
0619 12:08:43.593122 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.593123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.593134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.593195 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:43.593370 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.594101 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.595758 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.596354 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.596450 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.596459 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.596642 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.596766 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:08:43.596809 19912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.592726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:43115 (local address 127.0.203.131:40731)
0619 12:08:43.593009 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.593018 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.593073 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.593210 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.593211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.593212 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:43.593278 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.593279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.594226 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.594227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.595627 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.595631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.596461 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.596463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.596556 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.596564 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.596625 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.596685 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":30,"thread_start_us":177,"threads_started":1}
I20260619 12:08:43.596948 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.592806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:60427 (local address 127.0.203.130:40701)
0619 12:08:43.592852 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.592854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.592950 (+ 96us) server_negotiation.cc:407] Connection header received
0619 12:08:43.593024 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.593025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.593036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.593130 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:43.593336 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.594102 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.595759 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.596353 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.596456 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.596463 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.596642 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.596847 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:08:43.596951 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.592726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:40701 (local address 127.0.203.131:60427)
0619 12:08:43.592868 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.592884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:43.592953 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:43.593152 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:43.593153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:43.593155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:43.593251 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.593254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.594225 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.594226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.595628 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:43.595631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.596516 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.596516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:43.596573 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.596576 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:43.596625 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:08:43.596687 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":102}
I20260619 12:08:43.597349 19712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b0af12994264e02959417f464c8e583" candidate_uuid: "c5e4dfeb11924f4096897a1efc016e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc01445340b04985b3b591117da62309" is_pre_election: true
I20260619 12:08:43.597575 19712 raft_consensus.cc:2454] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5e4dfeb11924f4096897a1efc016e4f in term 0.
I20260619 12:08:43.597575 19581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b0af12994264e02959417f464c8e583" candidate_uuid: "c5e4dfeb11924f4096897a1efc016e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d514fd628e4a21b0a350caef89814a" is_pre_election: true
I20260619 12:08:43.597716 19581 raft_consensus.cc:2454] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5e4dfeb11924f4096897a1efc016e4f in term 0.
I20260619 12:08:43.597879 19777 leader_election.cc:302] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc01445340b04985b3b591117da62309, c5e4dfeb11924f4096897a1efc016e4f; no voters:
I20260619 12:08:43.598019 19911 raft_consensus.cc:2781] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:08:43.598119 19911 raft_consensus.cc:482] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:08:43.598142 19911 raft_consensus.cc:3037] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:43.598887 19911 raft_consensus.cc:504] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.599037 19911 leader_election.cc:288] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [CANDIDATE]: Term 1 election: Requested vote from peers c5d514fd628e4a21b0a350caef89814a (127.0.203.129:43115), bc01445340b04985b3b591117da62309 (127.0.203.130:40701)
I20260619 12:08:43.599221 19581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b0af12994264e02959417f464c8e583" candidate_uuid: "c5e4dfeb11924f4096897a1efc016e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d514fd628e4a21b0a350caef89814a"
I20260619 12:08:43.599287 19712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b0af12994264e02959417f464c8e583" candidate_uuid: "c5e4dfeb11924f4096897a1efc016e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc01445340b04985b3b591117da62309"
I20260619 12:08:43.599349 19581 raft_consensus.cc:3037] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:43.599393 19712 raft_consensus.cc:3037] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:43.600543 19581 raft_consensus.cc:2454] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5e4dfeb11924f4096897a1efc016e4f in term 1.
I20260619 12:08:43.600546 19712 raft_consensus.cc:2454] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5e4dfeb11924f4096897a1efc016e4f in term 1.
I20260619 12:08:43.600795 19779 leader_election.cc:302] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5d514fd628e4a21b0a350caef89814a, c5e4dfeb11924f4096897a1efc016e4f; no voters:
I20260619 12:08:43.600936 19911 raft_consensus.cc:2781] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:43.601143 19911 raft_consensus.cc:686] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 1 LEADER]: Becoming Leader. State: Replica: c5e4dfeb11924f4096897a1efc016e4f, State: Running, Role: LEADER
I20260619 12:08:43.601243 19911 consensus_queue.cc:237] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } }
I20260619 12:08:43.602244 19439 catalog_manager.cc:5495] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e4dfeb11924f4096897a1efc016e4f (127.0.203.131). New cstate: current_term: 1 leader_uuid: "c5e4dfeb11924f4096897a1efc016e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5e4dfeb11924f4096897a1efc016e4f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 } health_report { overall_health: UNKNOWN } } }
I20260619 12:08:43.627208 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.624220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.0.203.131:42639)
0619 12:08:43.624299 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:43.624301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:43.624312 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:43.624360 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:43.624361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:43.624386 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:43.624491 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:43.624709 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.625291 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.626174 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:43.626414 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:43.626481 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:43.626629 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:43.626744 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:08:43.626829 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:43.627054 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 12:08:43.627068 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:08:43.629685 19581 raft_consensus.cc:1261] T 7b0af12994264e02959417f464c8e583 P c5d514fd628e4a21b0a350caef89814a [term 1 FOLLOWER]: Refusing update from remote peer c5e4dfeb11924f4096897a1efc016e4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:43.629678 19712 raft_consensus.cc:1261] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 1 FOLLOWER]: Refusing update from remote peer c5e4dfeb11924f4096897a1efc016e4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:43.630085 19911 consensus_queue.cc:1035] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc01445340b04985b3b591117da62309" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 40701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:43.630267 19913 consensus_queue.cc:1035] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5d514fd628e4a21b0a350caef89814a" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:43.633030 19916 mvcc.cc:205] Tried to move back new op lower bound from 7298543303183970304 to 7298543303071522816. Current Snapshot: MvccSnapshot[applied={T|T < 7298543303183970304}]
I20260619 12:08:43.633606 19917 mvcc.cc:205] Tried to move back new op lower bound from 7298543303183970304 to 7298543303071522816. Current Snapshot: MvccSnapshot[applied={T|T < 7298543303183970304}]
I20260619 12:08:43.633852 19918 mvcc.cc:205] Tried to move back new op lower bound from 7298543303183970304 to 7298543303071522816. Current Snapshot: MvccSnapshot[applied={T|T < 7298543303183970304}]
I20260619 12:08:43.635735 19438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:43.635859 19438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:43.636773 19438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a67cf9519b94c3183caccb45d4c6a60: Sending DeleteTablet for 3 replicas of tablet 7b0af12994264e02959417f464c8e583
I20260619 12:08:43.637184 19561 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b0af12994264e02959417f464c8e583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:43 UTC) from {username='slave'} at 127.0.0.1:60932
I20260619 12:08:43.637202 19823 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b0af12994264e02959417f464c8e583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:43 UTC) from {username='slave'} at 127.0.0.1:59466
I20260619 12:08:43.637241 19692 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b0af12994264e02959417f464c8e583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:43 UTC) from {username='slave'} at 127.0.0.1:40540
I20260619 12:08:43.637480 19925 tablet_replica.cc:330] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: stopping tablet replica
I20260619 12:08:43.637574 19925 raft_consensus.cc:2229] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:43.637679 19925 raft_consensus.cc:2258] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:43.637872 19926 tablet_replica.cc:330] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: stopping tablet replica
I20260619 12:08:43.637953 19926 raft_consensus.cc:2229] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:08:43.637972 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 19499
I20260619 12:08:43.638044 19926 raft_consensus.cc:2258] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:43.638407 19925 ts_tablet_manager.cc:1905] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:43.640173 19925 ts_tablet_manager.cc:1918] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:43.640244 19925 log.cc:1196] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/7b0af12994264e02959417f464c8e583
I20260619 12:08:43.640473 19925 ts_tablet_manager.cc:1939] T 7b0af12994264e02959417f464c8e583 P bc01445340b04985b3b591117da62309: Deleting consensus metadata
I20260619 12:08:43.641548 19424 catalog_manager.cc:4841] TS bc01445340b04985b3b591117da62309 (127.0.203.130:40701): tablet 7b0af12994264e02959417f464c8e583 (table test-table [id=197016a27cec43e5a2f3f2de47ed8694]) successfully deleted
I20260619 12:08:43.643937 19926 ts_tablet_manager.cc:1905] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:08:43.644848 19426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:43115 (error 108)
I20260619 12:08:43.645092 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 19630
I20260619 12:08:43.645917 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:43.645267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:43115 (local address 127.0.0.1:60946)
0619 12:08:43.645833 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:43.645878 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:43115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541}
W20260619 12:08:43.646697 19426 catalog_manager.cc:4584] TS c5d514fd628e4a21b0a350caef89814a (127.0.203.129:43115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b0af12994264e02959417f464c8e583: Network error: Client connection negotiation failed: client connection to 127.0.203.129:43115: connect: Connection refused (error 111)
I20260619 12:08:43.647188 19926 ts_tablet_manager.cc:1918] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:43.647320 19926 log.cc:1196] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/7b0af12994264e02959417f464c8e583
I20260619 12:08:43.647677 19926 ts_tablet_manager.cc:1939] T 7b0af12994264e02959417f464c8e583 P c5e4dfeb11924f4096897a1efc016e4f: Deleting consensus metadata
I20260619 12:08:43.648258 19423 catalog_manager.cc:4841] TS c5e4dfeb11924f4096897a1efc016e4f (127.0.203.131:42639): tablet 7b0af12994264e02959417f464c8e583 (table test-table [id=197016a27cec43e5a2f3f2de47ed8694]) successfully deleted
I20260619 12:08:43.649928 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 19761
I20260619 12:08:43.654762 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 19408
2026-06-19T12:08:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (841 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:43 dist-test-slave-44vj krb5kdc[19931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:43 dist-test-slave-44vj krb5kdc[19931](info): set up 2 sockets
Jun 19 12:08:43 dist-test-slave-44vj krb5kdc[19931](info): commencing operation
krb5kdc: starting...
W20260619 12:08:44.631309 814 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:44 dist-test-slave-44vj krb5kdc[19931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:08:44.756335 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41923
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:41923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:44.846129 19944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:44.846446 19944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:44.846487 19944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:44.847862 19944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:44.847914 19944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:44.847932 19944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:44.847949 19944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:44.847962 19944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:44.849632 19944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:41923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:41923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:44.849920 19944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:44.850169 19944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:44.855577 19950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:44.855618 19949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:44.855679 19944 server_base.cc:1034] running on GCE node
W20260619 12:08:44.855577 19952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:44.856158 19944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:44.856488 19944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:44.857677 19944 hybrid_clock.cc:648] HybridClock initialized: now 1781870924857657 us; error 38 us; skew 500 ppm
Jun 19 12:08:44 dist-test-slave-44vj krb5kdc[19931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870924, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:44.860737 19944 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:08:44.861300 19944 webserver.cc:460] Webserver started at http://127.0.203.190:45661/ using document root <none> and password file <none>
I20260619 12:08:44.861531 19944 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:44.861581 19944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:44.861730 19944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:44.862952 19944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "af4b8b87aa534cfdb82c17b86710da5d"
format_stamp: "Formatted at 2026-06-19 12:08:44 on dist-test-slave-44vj"
server_key: "682225969a687ea35da7c46e30aa60b6"
server_key_iv: "5e0ac8f45621259e26a1bb30c72e6e8b"
server_key_version: "encryptionkey@0"
I20260619 12:08:44.863335 19944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "af4b8b87aa534cfdb82c17b86710da5d"
format_stamp: "Formatted at 2026-06-19 12:08:44 on dist-test-slave-44vj"
server_key: "682225969a687ea35da7c46e30aa60b6"
server_key_iv: "5e0ac8f45621259e26a1bb30c72e6e8b"
server_key_version: "encryptionkey@0"
I20260619 12:08:44.864897 19944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:44.865778 19959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:44.866010 19944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:44.866102 19944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "af4b8b87aa534cfdb82c17b86710da5d"
format_stamp: "Formatted at 2026-06-19 12:08:44 on dist-test-slave-44vj"
server_key: "682225969a687ea35da7c46e30aa60b6"
server_key_iv: "5e0ac8f45621259e26a1bb30c72e6e8b"
server_key_version: "encryptionkey@0"
I20260619 12:08:44.866199 19944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:44.883627 19944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:44.885874 19944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:44.886020 19944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:44.890799 19944 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:41923
I20260619 12:08:44.890874 20011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:41923 every 8 connection(s)
I20260619 12:08:44.891268 19944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:44.892252 20012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:44.892333 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 19944
I20260619 12:08:44.892457 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:44.892645 814 external_mini_cluster.cc:1428] Setting key 42080fbcb0425489778dee441a804a9c
I20260619 12:08:44.895102 20012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d: Bootstrap starting.
I20260619 12:08:44.895807 20012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:44.896099 20012 log.cc:824] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:44.896756 20012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d: No bootstrap required, opened a new log
I20260619 12:08:44.898873 20012 raft_consensus.cc:348] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4b8b87aa534cfdb82c17b86710da5d" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41923 } }
I20260619 12:08:44.899031 20012 raft_consensus.cc:374] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:44.899066 20012 raft_consensus.cc:729] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4b8b87aa534cfdb82c17b86710da5d, State: Initialized, Role: FOLLOWER
I20260619 12:08:44.899209 20012 consensus_queue.cc:260] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4b8b87aa534cfdb82c17b86710da5d" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41923 } }
I20260619 12:08:44.899287 20012 raft_consensus.cc:388] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:44.899325 20012 raft_consensus.cc:482] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:44.899406 20012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 0 FOLLOWER]: Advancing to term 1
Jun 19 12:08:44 dist-test-slave-44vj krb5kdc[19931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:44.900153 20012 raft_consensus.cc:504] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4b8b87aa534cfdb82c17b86710da5d" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41923 } }
I20260619 12:08:44.900261 20012 leader_election.cc:302] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af4b8b87aa534cfdb82c17b86710da5d; no voters:
I20260619 12:08:44.900465 20012 leader_election.cc:288] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:44.900547 20017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:44.900758 20017 raft_consensus.cc:686] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [term 1 LEADER]: Becoming Leader. State: Replica: af4b8b87aa534cfdb82c17b86710da5d, State: Running, Role: LEADER
I20260619 12:08:44.900791 20012 sys_catalog.cc:564] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:44.900877 20017 consensus_queue.cc:237] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4b8b87aa534cfdb82c17b86710da5d" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41923 } }
I20260619 12:08:44.901538 20019 sys_catalog.cc:455] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader af4b8b87aa534cfdb82c17b86710da5d. Latest consensus state: current_term: 1 leader_uuid: "af4b8b87aa534cfdb82c17b86710da5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4b8b87aa534cfdb82c17b86710da5d" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41923 } } }
I20260619 12:08:44.901577 20018 sys_catalog.cc:455] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af4b8b87aa534cfdb82c17b86710da5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4b8b87aa534cfdb82c17b86710da5d" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 41923 } } }
I20260619 12:08:44.901692 20018 sys_catalog.cc:458] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:44.901996 20026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:44.902209 20019 sys_catalog.cc:458] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:44.902731 20026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:44.905025 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:44.893332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.0.203.190:41923)
0619 12:08:44.893609 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:44.893620 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:44.893654 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:08:44.894264 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:44.894274 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:44.894329 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:44.894472 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:44.895132 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:44.896207 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:44.896979 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:44.897468 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:44.897574 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:44.900324 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:44.900329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:44.900329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:44.900344 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:44.903047 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:44.903569 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:44.903573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:44.903573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:44.903694 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:44.903923 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:44.903924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:44.903924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:44.904193 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:44.904354 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:44.904594 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 12:08:44.904745 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260619 12:08:44.905691 20026 catalog_manager.cc:1347] Generated new cluster ID: a3f940f67b4c497a92edfee5b5fdfe7f
I20260619 12:08:44.905753 20026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:44.951320 20026 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:44.951929 20026 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:44.954713 20026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d: Generated new TSK 0
I20260619 12:08:44.954916 20026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.203.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
Entry for principal kudu/127.0.203.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab.
I20260619 12:08:44.988696 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41923
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:45.081308 20038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:45.081564 20038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:45.081597 20038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:45.082852 20038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:45.082907 20038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:45.082945 20038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:45.084808 20038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.129.keytab
--principal=kudu/127.0.203.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:45.085103 20038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:45.085355 20038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:45.091086 20043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:45.091120 20044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:45.091091 20046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:45.092072 20038 server_base.cc:1034] running on GCE node
I20260619 12:08:45.092381 20038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:45.092739 20038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:45.093923 20038 hybrid_clock.cc:648] HybridClock initialized: now 1781870925093891 us; error 71 us; skew 500 ppm
Jun 19 12:08:45 dist-test-slave-44vj krb5kdc[19931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:45.097436 20038 init.cc:377] Logged in from keytab as kudu/127.0.203.129@KRBTEST.COM (short username kudu)
I20260619 12:08:45.098050 20038 webserver.cc:460] Webserver started at http://127.0.203.129:45465/ using document root <none> and password file <none>
I20260619 12:08:45.098304 20038 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:45.098356 20038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:45.098496 20038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:45.099861 20038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "fa557812a5ad4ae8bc58cabd84fe09cb"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "4b51b0b90cf0c706c6d51fa6d5992abc"
server_key_iv: "f73e3eb3978708205253ca78d06b53b6"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.100241 20038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "fa557812a5ad4ae8bc58cabd84fe09cb"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "4b51b0b90cf0c706c6d51fa6d5992abc"
server_key_iv: "f73e3eb3978708205253ca78d06b53b6"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.102061 20038 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:45.103070 20053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.103283 20038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:45.103401 20038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "fa557812a5ad4ae8bc58cabd84fe09cb"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "4b51b0b90cf0c706c6d51fa6d5992abc"
server_key_iv: "f73e3eb3978708205253ca78d06b53b6"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.103471 20038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:45.119664 20038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:45.122071 20038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:45.122221 20038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:45.122470 20038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:45.122835 20038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:45.122885 20038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.122920 20038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:45.122943 20038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.130053 20038 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:33143
I20260619 12:08:45.130143 20166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:33143 every 8 connection(s)
I20260619 12:08:45.130483 20038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:08:45.135697 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 20038
I20260619 12:08:45.135824 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:08:45.136047 814 external_mini_cluster.cc:1428] Setting key 617b9a9326daed2cecff358cffb30096
Jun 19 12:08:45 dist-test-slave-44vj krb5kdc[19931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.129@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:45.142427 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.131326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:34863 (local address 127.0.203.190:41923)
0619 12:08:45.131403 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.131405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.131659 (+ 254us) server_negotiation.cc:407] Connection header received
0619 12:08:45.132688 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.132690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.132701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.132783 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:45.133527 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.134172 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.135434 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.135714 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.135817 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.138805 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:45.138807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:45.138807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:45.138816 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:45.140503 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.141061 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.141063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.141064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.141168 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.141509 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.141510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.141510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.141679 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:45.141755 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.142194 (+ 439us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.142321 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:08:45.142797 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.131289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41923 (local address 127.0.203.129:34863)
0619 12:08:45.131513 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.131563 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.132574 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.132870 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.132874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.133178 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:45.133408 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.133412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.134323 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.134324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.135278 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.135281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.135874 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.135876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.136057 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.136724 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:45.136737 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:45.138678 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:45.140630 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:45.140645 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:45.140648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:45.140940 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:45.141277 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:45.141278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:45.141278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:45.141421 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:45.141778 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:45.141787 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:45.142009 (+ 222us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.142165 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.142409 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
I20260619 12:08:45.143476 20167 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41923
I20260619 12:08:45.143615 20167 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:45.143857 20167 heartbeater.cc:511] Master 127.0.203.190:41923 requested a full tablet report, sending...
I20260619 12:08:45.144585 19976 ts_manager.cc:194] Registered new tserver with Master: fa557812a5ad4ae8bc58cabd84fe09cb (127.0.203.129:33143)
I20260619 12:08:45.145521 19976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.129@KRBTEST.COM'} at 127.0.203.129:34863
WARNING: no policy specified for kudu/127.0.203.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
Entry for principal kudu/127.0.203.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab.
I20260619 12:08:45.163491 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41923
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:45.250413 20172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:45.250648 20172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:45.250672 20172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:45.252009 20172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:45.252068 20172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:45.252105 20172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:45.253744 20172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.130.keytab
--principal=kudu/127.0.203.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:45.253989 20172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:45.254218 20172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:45.259770 20180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:45.259778 20177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:45.260053 20172 server_base.cc:1034] running on GCE node
W20260619 12:08:45.259770 20178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:45.260366 20172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:45.260618 20172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:45.261814 20172 hybrid_clock.cc:648] HybridClock initialized: now 1781870925261785 us; error 41 us; skew 500 ppm
Jun 19 12:08:45 dist-test-slave-44vj krb5kdc[19931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:45.264722 20172 init.cc:377] Logged in from keytab as kudu/127.0.203.130@KRBTEST.COM (short username kudu)
I20260619 12:08:45.265241 20172 webserver.cc:460] Webserver started at http://127.0.203.130:36331/ using document root <none> and password file <none>
I20260619 12:08:45.265477 20172 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:45.265527 20172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:45.265627 20172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:45.266696 20172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "1f94326ed368436f81f0e2fa45323423"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "dc0145dffd0461098f2a3e42b588dac4"
server_key_iv: "269fe3dc45502a11335b10035c686077"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.267019 20172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "1f94326ed368436f81f0e2fa45323423"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "dc0145dffd0461098f2a3e42b588dac4"
server_key_iv: "269fe3dc45502a11335b10035c686077"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.268558 20172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:08:45.269495 20187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.269749 20172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:45.269866 20172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "1f94326ed368436f81f0e2fa45323423"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "dc0145dffd0461098f2a3e42b588dac4"
server_key_iv: "269fe3dc45502a11335b10035c686077"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.269946 20172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:45.282728 20172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:45.285189 20172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:45.285346 20172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:45.285604 20172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:45.285912 20172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:45.285948 20172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.285977 20172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:45.285995 20172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.292649 20172 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:46127
I20260619 12:08:45.292706 20300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:46127 every 8 connection(s)
I20260619 12:08:45.293040 20172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:45.299472 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 20172
Jun 19 12:08:45 dist-test-slave-44vj krb5kdc[19931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.130@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:45.299618 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:08:45.299858 814 external_mini_cluster.cc:1428] Setting key f62b6ff5d72e4b23a50014689fa2f0ee
I20260619 12:08:45.304107 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.293932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:42683 (local address 127.0.203.190:41923)
0619 12:08:45.294004 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.294006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.294186 (+ 180us) server_negotiation.cc:407] Connection header received
0619 12:08:45.295101 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.295102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.295113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.295197 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:45.295921 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.296477 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.297565 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.297764 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.297818 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.300430 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:45.300432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:45.300432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:45.300442 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:45.302261 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.302957 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.302959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.302959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.303052 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.303322 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.303323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.303323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.303515 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:45.303601 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.303907 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.304012 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:08:45.304288 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.293886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41923 (local address 127.0.203.130:42683)
0619 12:08:45.294047 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.294091 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.294998 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.295272 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.295277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.295618 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:45.295822 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.295827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.296596 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.296597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.297470 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.297473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.297834 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.297835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.297940 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.298488 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:45.298496 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:45.300283 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:45.302394 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:45.302408 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:45.302412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:45.302828 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:45.303130 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:45.303130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:45.303131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:45.303255 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:45.303590 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:45.303595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:45.303758 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.303862 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.304046 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260619 12:08:45.304874 20301 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41923
I20260619 12:08:45.305013 20301 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:45.305236 20301 heartbeater.cc:511] Master 127.0.203.190:41923 requested a full tablet report, sending...
I20260619 12:08:45.305691 19976 ts_manager.cc:194] Registered new tserver with Master: 1f94326ed368436f81f0e2fa45323423 (127.0.203.130:46127)
I20260619 12:08:45.306115 19976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.130@KRBTEST.COM'} at 127.0.203.130:42683
WARNING: no policy specified for kudu/127.0.203.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
Entry for principal kudu/127.0.203.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab.
I20260619 12:08:45.326311 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41923
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:45.412448 20306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:45.412684 20306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:45.412712 20306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:45.414003 20306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:45.414065 20306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:45.414098 20306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:45.416117 20306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/krb5kdc/kudu_127.0.203.131.keytab
--principal=kudu/127.0.203.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:41923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:45.416373 20306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:45.416580 20306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:45.422114 20312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:45.422098 20314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:45.422098 20311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:45.422483 20306 server_base.cc:1034] running on GCE node
I20260619 12:08:45.422740 20306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:45.423018 20306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:45.424180 20306 hybrid_clock.cc:648] HybridClock initialized: now 1781870925424141 us; error 53 us; skew 500 ppm
Jun 19 12:08:45 dist-test-slave-44vj krb5kdc[19931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:45.427088 20306 init.cc:377] Logged in from keytab as kudu/127.0.203.131@KRBTEST.COM (short username kudu)
I20260619 12:08:45.427662 20306 webserver.cc:460] Webserver started at http://127.0.203.131:38141/ using document root <none> and password file <none>
I20260619 12:08:45.427873 20306 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:45.427918 20306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:45.428020 20306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:45.429061 20306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "a725b4a0c43a4a208e74eb02f21a188f"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "a272d1c7629b7afd9316419a3ba29694"
server_key_iv: "e7afb96b2941126c06f9a5514aa48c30"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.429354 20306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "a725b4a0c43a4a208e74eb02f21a188f"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "a272d1c7629b7afd9316419a3ba29694"
server_key_iv: "e7afb96b2941126c06f9a5514aa48c30"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.430706 20306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:45.431608 20321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.431826 20306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:45.431933 20306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "a725b4a0c43a4a208e74eb02f21a188f"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "a272d1c7629b7afd9316419a3ba29694"
server_key_iv: "e7afb96b2941126c06f9a5514aa48c30"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.432008 20306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:45.440455 20306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:45.442543 20306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:45.442684 20306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:45.442910 20306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:45.443225 20306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:45.443274 20306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.443305 20306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:45.443327 20306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.449921 20306 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:37339
I20260619 12:08:45.450074 20434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:37339 every 8 connection(s)
I20260619 12:08:45.450345 20306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:08:45.451874 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 20306
I20260619 12:08:45.451973 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:08:45.452162 814 external_mini_cluster.cc:1428] Setting key 8858fbed48b150d7b93c6bb01188bcbe
Jun 19 12:08:45 dist-test-slave-44vj krb5kdc[19931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.131@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:08:45.461974 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.451267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:60397 (local address 127.0.203.190:41923)
0619 12:08:45.451389 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.451392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.451590 (+ 198us) server_negotiation.cc:407] Connection header received
0619 12:08:45.452996 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.452998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.453014 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.453069 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:45.453897 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.454445 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.455761 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.456074 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.456151 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.459262 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:45.459266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:45.459266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:45.459276 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:45.460706 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.461171 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.461172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.461172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.461227 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.461428 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.461429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.461429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.461541 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:45.461594 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.461841 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.461906 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:08:45.462268 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.451163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:41923 (local address 127.0.203.131:60397)
0619 12:08:45.451419 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.451478 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.452864 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.453165 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.453170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.453513 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:45.453763 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.453768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.454552 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.454553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.455670 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.455674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.456183 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.456184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.456353 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.457084 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:08:45.457096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:45.459139 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:45.460802 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:45.460809 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:45.460811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:45.461101 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:45.461277 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:08:45.461277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:08:45.461277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:08:45.461380 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:08:45.461594 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:45.461597 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:45.461730 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.461821 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.461963 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
I20260619 12:08:45.462837 20435 heartbeater.cc:347] Connected to a master server at 127.0.203.190:41923
I20260619 12:08:45.462936 20435 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:45.463167 20435 heartbeater.cc:511] Master 127.0.203.190:41923 requested a full tablet report, sending...
I20260619 12:08:45.463689 19976 ts_manager.cc:194] Registered new tserver with Master: a725b4a0c43a4a208e74eb02f21a188f (127.0.203.131:37339)
I20260619 12:08:45.464165 19976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.203.131@KRBTEST.COM'} at 127.0.203.131:60397
I20260619 12:08:45.464232 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:08:45.471319 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.465555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.0.203.190:41923)
0619 12:08:45.465597 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.465599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.465747 (+ 148us) server_negotiation.cc:407] Connection header received
0619 12:08:45.465808 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.465808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.465818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.465870 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:45.466386 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.466872 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.467618 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.467808 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.467865 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.468993 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:45.468995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:45.468995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:45.469002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:45.470283 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.470580 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.470581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.470582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.470634 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:45.470790 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:45.470790 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:45.470791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:45.470954 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:45.471044 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.471147 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.471228 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:08:45.473364 19976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:08:45.475545 19976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:08:45.484118 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.478577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:37339 (local address 127.0.0.1:47650)
0619 12:08:45.478934 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.478981 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.479056 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.479572 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.479573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.479576 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:45.479650 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.479653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.481267 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.481268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.482771 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.482776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.483746 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.483747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.483845 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.483865 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.483950 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.484017 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":27,"thread_start_us":103,"threads_started":1}
I20260619 12:08:45.484566 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.478575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:46127 (local address 127.0.0.1:54392)
0619 12:08:45.479031 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.479045 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.479106 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.481415 (+ 2309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.481416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.481419 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:45.481490 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.481492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.482360 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.482361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.483289 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.483292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.484336 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.484338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.484412 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.484417 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.484446 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.484495 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":2,"spinlock_wait_cycles":896,"thread_start_us":115,"threads_started":1}
I20260619 12:08:45.484581 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.478815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.0.203.131:37339)
0619 12:08:45.479246 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.479252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.479263 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:45.479312 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.479314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.479347 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.479465 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:45.479728 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.480988 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.482885 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.483653 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.483728 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.483797 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.484417 (+ 620us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.484499 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":108,"threads_started":1}
I20260619 12:08:45.484987 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.478796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54392 (local address 127.0.203.130:46127)
0619 12:08:45.481246 (+ 2450us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.481252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.481265 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:08:45.481310 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.481312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.481332 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.481418 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:45.481548 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.482261 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.483469 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.484221 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.484776 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.484851 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.484876 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.484927 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2402,"thread_start_us":91,"threads_started":1}
I20260619 12:08:45.485090 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.478585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33143 (local address 127.0.0.1:57378)
0619 12:08:45.479181 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.479190 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.479236 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.479540 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.479543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.479546 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:45.479628 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.479630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.481213 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.481216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.482771 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.482776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.484596 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.484597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.484870 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.484877 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.484936 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.484996 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":29,"thread_start_us":151,"threads_started":1}
I20260619 12:08:45.485366 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.478949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.0.203.129:33143)
0619 12:08:45.479143 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.479149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.479299 (+ 150us) server_negotiation.cc:407] Connection header received
0619 12:08:45.479348 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.479351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.479413 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.479488 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:45.479723 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.481060 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.483842 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.484518 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.484590 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.484646 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.485224 (+ 578us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.485280 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260619 12:08:45.485450 20369 tablet_service.cc:1467] Processing CreateTablet for tablet 89b7723bf9264b369c13e04bafb06153 (DEFAULT_TABLE table=test-table [id=e621200911e24c988befbe875e5ab4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:45.485730 20235 tablet_service.cc:1467] Processing CreateTablet for tablet 89b7723bf9264b369c13e04bafb06153 (DEFAULT_TABLE table=test-table [id=e621200911e24c988befbe875e5ab4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:45.485858 20369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b7723bf9264b369c13e04bafb06153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:45.486071 20235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b7723bf9264b369c13e04bafb06153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:45.488770 20452 tablet_bootstrap.cc:492] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Bootstrap starting.
I20260619 12:08:45.489264 20453 tablet_bootstrap.cc:492] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Bootstrap starting.
I20260619 12:08:45.489491 20452 tablet_bootstrap.cc:654] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:45.490000 20453 tablet_bootstrap.cc:654] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:45.489859 20101 tablet_service.cc:1467] Processing CreateTablet for tablet 89b7723bf9264b369c13e04bafb06153 (DEFAULT_TABLE table=test-table [id=e621200911e24c988befbe875e5ab4b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:45.490073 20452 log.cc:824] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:45.490264 20101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b7723bf9264b369c13e04bafb06153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:45.490520 20453 log.cc:824] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:45.491430 20453 tablet_bootstrap.cc:492] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: No bootstrap required, opened a new log
I20260619 12:08:45.491526 20453 ts_tablet_manager.cc:1397] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:45.491710 20452 tablet_bootstrap.cc:492] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: No bootstrap required, opened a new log
I20260619 12:08:45.491786 20452 ts_tablet_manager.cc:1397] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:08:45.493155 20452 raft_consensus.cc:348] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.493311 20452 raft_consensus.cc:374] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:45.493345 20452 raft_consensus.cc:729] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f94326ed368436f81f0e2fa45323423, State: Initialized, Role: FOLLOWER
I20260619 12:08:45.493533 20452 consensus_queue.cc:260] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.493813 20301 heartbeater.cc:503] Master 127.0.203.190:41923 was elected leader, sending a full tablet report...
I20260619 12:08:45.493712 20453 raft_consensus.cc:348] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.493901 20453 raft_consensus.cc:374] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:45.493935 20453 raft_consensus.cc:729] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a725b4a0c43a4a208e74eb02f21a188f, State: Initialized, Role: FOLLOWER
I20260619 12:08:45.494040 20457 tablet_bootstrap.cc:492] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: Bootstrap starting.
I20260619 12:08:45.493826 20452 ts_tablet_manager.cc:1428] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:45.494253 20453 consensus_queue.cc:260] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.494696 20453 ts_tablet_manager.cc:1428] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:08:45.494786 20457 tablet_bootstrap.cc:654] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:45.494827 20435 heartbeater.cc:503] Master 127.0.203.190:41923 was elected leader, sending a full tablet report...
I20260619 12:08:45.495410 20457 log.cc:824] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:45.496150 20457 tablet_bootstrap.cc:492] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: No bootstrap required, opened a new log
I20260619 12:08:45.496217 20457 ts_tablet_manager.cc:1397] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:45.497669 20457 raft_consensus.cc:348] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.497829 20457 raft_consensus.cc:374] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:45.497861 20457 raft_consensus.cc:729] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa557812a5ad4ae8bc58cabd84fe09cb, State: Initialized, Role: FOLLOWER
I20260619 12:08:45.497982 20457 consensus_queue.cc:260] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.498253 20457 ts_tablet_manager.cc:1428] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:08:45.498353 20167 heartbeater.cc:503] Master 127.0.203.190:41923 was elected leader, sending a full tablet report...
W20260619 12:08:45.543744 20302 tablet.cc:2369] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:08:45.631418 20168 tablet.cc:2369] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:45.655865 20458 raft_consensus.cc:482] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:45.656016 20458 raft_consensus.cc:504] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.656471 20458 leader_election.cc:288] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f94326ed368436f81f0e2fa45323423 (127.0.203.130:46127), fa557812a5ad4ae8bc58cabd84fe09cb (127.0.203.129:33143)
I20260619 12:08:45.659965 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.656894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:44575 (local address 127.0.203.130:46127)
0619 12:08:45.656952 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.656953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.656964 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:45.657007 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.657008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.657019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.657128 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:45.657352 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.657907 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.659090 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.659570 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.659638 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.659646 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.659837 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.659883 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:08:45.659958 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.656755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:46127 (local address 127.0.203.131:44575)
0619 12:08:45.656839 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.656853 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.656923 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.657151 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.657152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.657155 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:45.657265 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.657268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.657990 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.657991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.658958 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.658961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.659656 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.659657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.659753 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.659758 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.659807 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.659855 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":60}
I20260619 12:08:45.660024 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.656862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:35499 (local address 127.0.203.129:33143)
0619 12:08:45.656931 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.656933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.657137 (+ 204us) server_negotiation.cc:407] Connection header received
0619 12:08:45.657183 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.657185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.657196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.657273 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:45.657472 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.658054 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.659196 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.659642 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.659721 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.659733 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.659906 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.659950 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:08:45.660293 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.656755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33143 (local address 127.0.203.131:35499)
0619 12:08:45.657043 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.657059 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:45.657128 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:45.657282 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:45.657283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:45.657285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:45.657389 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.657391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.658177 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.658179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.659086 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:45.659089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.659739 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.659740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:45.659820 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.659827 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:08:45.659892 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:08:45.659946 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":44,"thread_start_us":138,"threads_started":1}
I20260619 12:08:45.660462 20255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b7723bf9264b369c13e04bafb06153" candidate_uuid: "a725b4a0c43a4a208e74eb02f21a188f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f94326ed368436f81f0e2fa45323423" is_pre_election: true
I20260619 12:08:45.660699 20255 raft_consensus.cc:2454] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a725b4a0c43a4a208e74eb02f21a188f in term 0.
I20260619 12:08:45.660719 20121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b7723bf9264b369c13e04bafb06153" candidate_uuid: "a725b4a0c43a4a208e74eb02f21a188f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" is_pre_election: true
I20260619 12:08:45.660929 20121 raft_consensus.cc:2454] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a725b4a0c43a4a208e74eb02f21a188f in term 0.
I20260619 12:08:45.660938 20325 leader_election.cc:302] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f94326ed368436f81f0e2fa45323423, a725b4a0c43a4a208e74eb02f21a188f; no voters:
I20260619 12:08:45.661100 20458 raft_consensus.cc:2781] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:08:45.661183 20458 raft_consensus.cc:482] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:08:45.661206 20458 raft_consensus.cc:3037] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:45.661947 20458 raft_consensus.cc:504] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.662081 20458 leader_election.cc:288] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [CANDIDATE]: Term 1 election: Requested vote from peers 1f94326ed368436f81f0e2fa45323423 (127.0.203.130:46127), fa557812a5ad4ae8bc58cabd84fe09cb (127.0.203.129:33143)
I20260619 12:08:45.662319 20121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b7723bf9264b369c13e04bafb06153" candidate_uuid: "a725b4a0c43a4a208e74eb02f21a188f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb"
I20260619 12:08:45.662413 20121 raft_consensus.cc:3037] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:45.662417 20255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b7723bf9264b369c13e04bafb06153" candidate_uuid: "a725b4a0c43a4a208e74eb02f21a188f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f94326ed368436f81f0e2fa45323423"
I20260619 12:08:45.662544 20255 raft_consensus.cc:3037] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:45.663264 20121 raft_consensus.cc:2454] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a725b4a0c43a4a208e74eb02f21a188f in term 1.
I20260619 12:08:45.663497 20324 leader_election.cc:302] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a725b4a0c43a4a208e74eb02f21a188f, fa557812a5ad4ae8bc58cabd84fe09cb; no voters:
I20260619 12:08:45.663609 20458 raft_consensus.cc:2781] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:45.663770 20458 raft_consensus.cc:686] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 1 LEADER]: Becoming Leader. State: Replica: a725b4a0c43a4a208e74eb02f21a188f, State: Running, Role: LEADER
I20260619 12:08:45.663801 20255 raft_consensus.cc:2454] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a725b4a0c43a4a208e74eb02f21a188f in term 1.
I20260619 12:08:45.663888 20458 consensus_queue.cc:237] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } }
I20260619 12:08:45.664917 19975 catalog_manager.cc:5495] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f reported cstate change: term changed from 0 to 1, leader changed from <none> to a725b4a0c43a4a208e74eb02f21a188f (127.0.203.131). New cstate: current_term: 1 leader_uuid: "a725b4a0c43a4a208e74eb02f21a188f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a725b4a0c43a4a208e74eb02f21a188f" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 37339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 } health_report { overall_health: UNKNOWN } } }
W20260619 12:08:45.701084 20436 tablet.cc:2369] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:45.714124 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.711248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.0.203.131:37339)
0619 12:08:45.711306 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.711308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.711318 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:08:45.711399 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.711400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.711421 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.711509 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:45.711702 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.712305 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.713282 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.713494 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.713580 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.713746 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:45.713874 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:08:45.713952 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.713999 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.714021 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:08:45.716552 20121 raft_consensus.cc:1261] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 1 FOLLOWER]: Refusing update from remote peer a725b4a0c43a4a208e74eb02f21a188f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:45.716562 20255 raft_consensus.cc:1261] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 1 FOLLOWER]: Refusing update from remote peer a725b4a0c43a4a208e74eb02f21a188f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:45.716970 20458 consensus_queue.cc:1035] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa557812a5ad4ae8bc58cabd84fe09cb" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 33143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:45.717193 20462 consensus_queue.cc:1035] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f94326ed368436f81f0e2fa45323423" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:45.723529 19975 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:45.723652 19975 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:45.724592 19975 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af4b8b87aa534cfdb82c17b86710da5d: Sending DeleteTablet for 3 replicas of tablet 89b7723bf9264b369c13e04bafb06153
I20260619 12:08:45.724963 20235 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b7723bf9264b369c13e04bafb06153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:45 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:54392
I20260619 12:08:45.724969 20369 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b7723bf9264b369c13e04bafb06153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:45 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:47650
I20260619 12:08:45.725095 20101 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b7723bf9264b369c13e04bafb06153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:45 UTC) from {username='kudu', principal='kudu/127.0.203.190@KRBTEST.COM'} at 127.0.0.1:57378
I20260619 12:08:45.725330 20476 tablet_replica.cc:330] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: stopping tablet replica
I20260619 12:08:45.725466 20476 raft_consensus.cc:2229] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:08:45.725627 20476 raft_consensus.cc:2258] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:45.725741 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 20038
I20260619 12:08:45.725807 20477 tablet_replica.cc:330] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb: stopping tablet replica
I20260619 12:08:45.725912 20477 raft_consensus.cc:2229] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:45.726024 20477 raft_consensus.cc:2258] T 89b7723bf9264b369c13e04bafb06153 P fa557812a5ad4ae8bc58cabd84fe09cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:45.726810 20476 ts_tablet_manager.cc:1905] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:45.728621 20476 ts_tablet_manager.cc:1918] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:45.728698 20476 log.cc:1196] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/89b7723bf9264b369c13e04bafb06153
I20260619 12:08:45.728960 20476 ts_tablet_manager.cc:1939] T 89b7723bf9264b369c13e04bafb06153 P a725b4a0c43a4a208e74eb02f21a188f: Deleting consensus metadata
I20260619 12:08:45.729382 19960 catalog_manager.cc:4841] TS a725b4a0c43a4a208e74eb02f21a188f (127.0.203.131:37339): tablet 89b7723bf9264b369c13e04bafb06153 (table test-table [id=e621200911e24c988befbe875e5ab4b7]) successfully deleted
I20260619 12:08:45.729892 20475 tablet_replica.cc:330] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: stopping tablet replica
I20260619 12:08:45.730027 20475 raft_consensus.cc:2229] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:45.730141 20475 raft_consensus.cc:2258] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:45.730683 20475 ts_tablet_manager.cc:1905] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:45.732286 20475 ts_tablet_manager.cc:1918] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:45.732365 20475 log.cc:1196] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/89b7723bf9264b369c13e04bafb06153
I20260619 12:08:45.732601 20475 ts_tablet_manager.cc:1939] T 89b7723bf9264b369c13e04bafb06153 P 1f94326ed368436f81f0e2fa45323423: Deleting consensus metadata
W20260619 12:08:45.733105 19962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:33143 (error 108)
I20260619 12:08:45.733244 19963 catalog_manager.cc:4841] TS 1f94326ed368436f81f0e2fa45323423 (127.0.203.130:46127): tablet 89b7723bf9264b369c13e04bafb06153 (table test-table [id=e621200911e24c988befbe875e5ab4b7]) successfully deleted
I20260619 12:08:45.733763 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.733613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:33143 (local address 127.0.0.1:41664)
0619 12:08:45.733674 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:45.733717 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 12:08:45.733749 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 20172
W20260619 12:08:45.736440 19962 catalog_manager.cc:4584] TS fa557812a5ad4ae8bc58cabd84fe09cb (127.0.203.129:33143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89b7723bf9264b369c13e04bafb06153: Network error: Client connection negotiation failed: client connection to 127.0.203.129:33143: connect: Connection refused (error 111)
I20260619 12:08:45.739812 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 20306
I20260619 12:08:45.745223 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 19944
2026-06-19T12:08:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2092 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T12:08:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:45Z Disabled control of system clock
I20260619 12:08:45.775943 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34455
--webserver_interface=127.0.203.190
--webserver_port=0
--builtin_ntp_servers=127.0.203.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:45.866257 20482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:45.866549 20482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:45.866570 20482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:45.867942 20482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:45.867996 20482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:45.868008 20482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:45.868021 20482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:45.868031 20482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:45.869516 20482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:34455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:45.869799 20482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:45.870129 20482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:45.875434 20488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:45.875488 20487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:45.875583 20482 server_base.cc:1034] running on GCE node
W20260619 12:08:45.875488 20490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:45.875974 20482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:45.876358 20482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:45.877529 20482 hybrid_clock.cc:648] HybridClock initialized: now 1781870925877505 us; error 51 us; skew 500 ppm
I20260619 12:08:45.879231 20482 webserver.cc:460] Webserver started at http://127.0.203.190:36469/ using document root <none> and password file <none>
I20260619 12:08:45.879565 20482 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:45.879626 20482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:45.879755 20482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:45.881047 20482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "80abe9d83652477da0f773a124a932da"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "2d89fd53b3965fa8a1b88cc4a72e4e24"
server_key_iv: "8e21a908e3b6f5473f62640546b5f8fa"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.881408 20482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "80abe9d83652477da0f773a124a932da"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "2d89fd53b3965fa8a1b88cc4a72e4e24"
server_key_iv: "8e21a908e3b6f5473f62640546b5f8fa"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.882917 20482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:08:45.883850 20496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:45.884102 20482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:08:45.884203 20482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "80abe9d83652477da0f773a124a932da"
format_stamp: "Formatted at 2026-06-19 12:08:45 on dist-test-slave-44vj"
server_key: "2d89fd53b3965fa8a1b88cc4a72e4e24"
server_key_iv: "8e21a908e3b6f5473f62640546b5f8fa"
server_key_version: "encryptionkey@0"
I20260619 12:08:45.884271 20482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:45.911429 20482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:45.911851 20482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:45.912002 20482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:45.916819 20482 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:34455
I20260619 12:08:45.916913 20548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:34455 every 8 connection(s)
I20260619 12:08:45.917279 20482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:45.918432 20549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:45.921082 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da: Bootstrap starting.
I20260619 12:08:45.921723 20549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:45.922006 20549 log.cc:824] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:45.922514 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 20482
I20260619 12:08:45.922643 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:45.922832 814 external_mini_cluster.cc:1428] Setting key 07a3d77999bc75828b92a6ee8d04640e
I20260619 12:08:45.923081 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da: No bootstrap required, opened a new log
I20260619 12:08:45.925539 20549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80abe9d83652477da0f773a124a932da" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34455 } }
I20260619 12:08:45.925777 20549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:45.925817 20549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80abe9d83652477da0f773a124a932da, State: Initialized, Role: FOLLOWER
I20260619 12:08:45.925982 20549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80abe9d83652477da0f773a124a932da" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34455 } }
I20260619 12:08:45.926067 20549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:45.926111 20549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:45.926164 20549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:45.926985 20549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80abe9d83652477da0f773a124a932da" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34455 } }
I20260619 12:08:45.927110 20549 leader_election.cc:302] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80abe9d83652477da0f773a124a932da; no voters:
I20260619 12:08:45.927313 20549 leader_election.cc:288] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:45.927351 20554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:45.927492 20554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [term 1 LEADER]: Becoming Leader. State: Replica: 80abe9d83652477da0f773a124a932da, State: Running, Role: LEADER
I20260619 12:08:45.927585 20554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80abe9d83652477da0f773a124a932da" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34455 } }
I20260619 12:08:45.927670 20549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:45.928010 20554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80abe9d83652477da0f773a124a932da. Latest consensus state: current_term: 1 leader_uuid: "80abe9d83652477da0f773a124a932da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80abe9d83652477da0f773a124a932da" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34455 } } }
I20260619 12:08:45.928124 20554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:45.928011 20555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80abe9d83652477da0f773a124a932da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80abe9d83652477da0f773a124a932da" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 34455 } } }
I20260619 12:08:45.928210 20555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:45.928370 20560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:45.928982 20560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:45.931087 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:45.923593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.0.203.190:34455)
0619 12:08:45.923925 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:45.923936 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:45.923981 (+ 45us) server_negotiation.cc:407] Connection header received
0619 12:08:45.924579 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:45.924590 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:45.924687 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:45.924876 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:45.925128 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.926954 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.928217 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:45.928957 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:45.929066 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:45.930215 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:45.930220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:45.930221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:45.930472 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:45.930529 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:45.930546 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:45.930648 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:45.930742 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 12:08:45.930850 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":147,"threads_started":1}
I20260619 12:08:45.931341 20560 catalog_manager.cc:1347] Generated new cluster ID: a312a22c1ef24f9e895043813b87e4b7
I20260619 12:08:45.931428 20560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:45.938035 20560 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:45.938623 20560 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:45.947196 20560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da: Generated new TSK 0
I20260619 12:08:45.947556 20560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:45.950289 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.129:0
--local_ip_for_outbound_sockets=127.0.203.129
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34455
--builtin_ntp_servers=127.0.203.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:08:46.039784 20573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:46.040036 20573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:46.040057 20573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:46.041256 20573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:46.041302 20573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:46.041327 20573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.129
I20260619 12:08:46.042716 20573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.203.129
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.129
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:46.043002 20573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:46.043413 20573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:46.049214 20578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:46.049432 20573 server_base.cc:1034] running on GCE node
W20260619 12:08:46.049237 20579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:46.049252 20581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:46.049902 20573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:46.050257 20573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:46.051447 20573 hybrid_clock.cc:648] HybridClock initialized: now 1781870926051412 us; error 54 us; skew 500 ppm
I20260619 12:08:46.053155 20573 webserver.cc:460] Webserver started at http://127.0.203.129:46785/ using document root <none> and password file <none>
I20260619 12:08:46.053474 20573 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:46.053560 20573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:46.053705 20573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:46.055243 20573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data/instance:
uuid: "6f37d698cfbf40aeb6d0c684d2855ddf"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "b9cc1f7ffb79531dfcbaea5fc98240b6"
server_key_iv: "17a1a9af81f951dbe0cbe28e2e096cde"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.055727 20573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/wal/instance:
uuid: "6f37d698cfbf40aeb6d0c684d2855ddf"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "b9cc1f7ffb79531dfcbaea5fc98240b6"
server_key_iv: "17a1a9af81f951dbe0cbe28e2e096cde"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.057603 20573 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:08:46.058575 20587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.058789 20573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:46.058909 20573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/wal
uuid: "6f37d698cfbf40aeb6d0c684d2855ddf"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "b9cc1f7ffb79531dfcbaea5fc98240b6"
server_key_iv: "17a1a9af81f951dbe0cbe28e2e096cde"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.058995 20573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:46.070564 20573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:46.070866 20573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:46.070981 20573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:46.071198 20573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:46.071555 20573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:46.071593 20573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.071632 20573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:46.071661 20573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.079010 20573 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.129:39509
I20260619 12:08:46.079087 20700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.129:39509 every 8 connection(s)
I20260619 12:08:46.079639 20573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/data/info.pb
I20260619 12:08:46.086233 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.080943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.129:39047 (local address 127.0.203.190:34455)
0619 12:08:46.081106 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.081109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.081331 (+ 222us) server_negotiation.cc:407] Connection header received
0619 12:08:46.082044 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.082045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.082067 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.082126 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:46.082942 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.083585 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.084684 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.084903 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.084962 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.085928 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:46.085930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:46.085930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:46.085938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:46.085956 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:46.085960 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:46.086008 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.086093 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.086125 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260619 12:08:46.086503 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.080871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34455 (local address 127.0.203.129:39047)
0619 12:08:46.081154 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.081211 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.081908 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.082217 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.082222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.082232 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:46.082845 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.082849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.083691 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.083693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.084600 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.084603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.084977 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.084978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.085122 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.085775 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:46.085775 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:46.085812 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:46.085842 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:46.086013 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:46.086019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:46.086022 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.086093 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.086220 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
I20260619 12:08:46.086551 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 20573
I20260619 12:08:46.086663 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-0/wal/instance
I20260619 12:08:46.086848 814 external_mini_cluster.cc:1428] Setting key 93e63555d1537937d690c075e3a86a9c
I20260619 12:08:46.087224 20701 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34455
I20260619 12:08:46.087340 20701 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:46.087653 20701 heartbeater.cc:511] Master 127.0.203.190:34455 requested a full tablet report, sending...
I20260619 12:08:46.087929 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.130:0
--local_ip_for_outbound_sockets=127.0.203.130
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34455
--builtin_ntp_servers=127.0.203.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:08:46.088315 20513 ts_manager.cc:194] Registered new tserver with Master: 6f37d698cfbf40aeb6d0c684d2855ddf (127.0.203.129:39509)
I20260619 12:08:46.089196 20513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.129:39047
W20260619 12:08:46.176441 20704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:46.176694 20704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:46.176723 20704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:46.177946 20704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:46.177989 20704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:46.178030 20704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.130
I20260619 12:08:46.179436 20704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.203.130
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.130
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:46.179708 20704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:46.180030 20704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:46.185549 20709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:46.185678 20710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:46.185779 20704 server_base.cc:1034] running on GCE node
W20260619 12:08:46.185904 20712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:46.186185 20704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:46.186499 20704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:46.187737 20704 hybrid_clock.cc:648] HybridClock initialized: now 1781870926187713 us; error 51 us; skew 500 ppm
I20260619 12:08:46.189273 20704 webserver.cc:460] Webserver started at http://127.0.203.130:40967/ using document root <none> and password file <none>
I20260619 12:08:46.189533 20704 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:46.189590 20704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:46.189736 20704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:46.190922 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data/instance:
uuid: "2c3b5d0d2c50478cb15b122c04b1dc47"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "81c4298dbedb01215727470625039d9b"
server_key_iv: "4e932064b84fa19f98715b6a17ffd8b8"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.191277 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal/instance:
uuid: "2c3b5d0d2c50478cb15b122c04b1dc47"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "81c4298dbedb01215727470625039d9b"
server_key_iv: "4e932064b84fa19f98715b6a17ffd8b8"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.192685 20704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:46.193495 20718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.193737 20704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:46.193828 20704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal
uuid: "2c3b5d0d2c50478cb15b122c04b1dc47"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "81c4298dbedb01215727470625039d9b"
server_key_iv: "4e932064b84fa19f98715b6a17ffd8b8"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.193894 20704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:46.202760 20704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:46.203094 20704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:46.203222 20704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:46.203545 20704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:46.204068 20704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:46.204113 20704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.204151 20704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:46.204169 20704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.210850 20704 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.130:46879
I20260619 12:08:46.210911 20831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.130:46879 every 8 connection(s)
I20260619 12:08:46.211481 20704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/data/info.pb
I20260619 12:08:46.214133 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 20704
I20260619 12:08:46.214252 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal/instance
I20260619 12:08:46.214488 814 external_mini_cluster.cc:1428] Setting key abee03a794f12b0b7d0d6d2c0f29b7b1
I20260619 12:08:46.215694 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.203.131:0
--local_ip_for_outbound_sockets=127.0.203.131
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34455
--builtin_ntp_servers=127.0.203.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:08:46.218138 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.212471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.130:50321 (local address 127.0.203.190:34455)
0619 12:08:46.212609 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.212611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.212850 (+ 239us) server_negotiation.cc:407] Connection header received
0619 12:08:46.213517 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.213518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.213532 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.213599 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:46.214351 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.215074 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.216378 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.216663 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.216754 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.217852 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:46.217855 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:46.217855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:46.217861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:46.217878 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:46.217883 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:46.217927 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.218039 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.218067 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 12:08:46.218402 20834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.212393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34455 (local address 127.0.203.130:50321)
0619 12:08:46.212702 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.212745 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.213370 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.213691 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.213695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.213704 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:46.214226 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.214231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.215208 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.215210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.216265 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.216269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.216827 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.216828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.216984 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.217670 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:46.217671 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:46.217714 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:46.217747 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:46.217948 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:46.217953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:46.217957 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.218035 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.218160 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":125,"threads_started":1}
I20260619 12:08:46.219007 20832 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34455
I20260619 12:08:46.219126 20832 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:46.219342 20832 heartbeater.cc:511] Master 127.0.203.190:34455 requested a full tablet report, sending...
I20260619 12:08:46.219799 20513 ts_manager.cc:194] Registered new tserver with Master: 2c3b5d0d2c50478cb15b122c04b1dc47 (127.0.203.130:46879)
I20260619 12:08:46.220327 20513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.130:50321
W20260619 12:08:46.303028 20835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:46.303274 20835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:46.303300 20835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:46.304726 20835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:46.304790 20835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:46.304819 20835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.203.131
I20260619 12:08:46.306290 20835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.203.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.203.131
--webserver_port=0
--tserver_master_addrs=127.0.203.190:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.203.131
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:46.306519 20835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:46.306819 20835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:46.312391 20843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:46.312595 20835 server_base.cc:1034] running on GCE node
W20260619 12:08:46.312394 20840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:46.312387 20841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:46.312997 20835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:46.313329 20835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:46.314512 20835 hybrid_clock.cc:648] HybridClock initialized: now 1781870926314500 us; error 41 us; skew 500 ppm
I20260619 12:08:46.316126 20835 webserver.cc:460] Webserver started at http://127.0.203.131:40327/ using document root <none> and password file <none>
I20260619 12:08:46.316388 20835 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:46.316442 20835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:46.316589 20835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:46.317842 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data/instance:
uuid: "6acc2c81b8294930bc96491e466c1c87"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "9f5dcd387758899cfe22804ffee887f6"
server_key_iv: "34b495e735be83c980eb236c7f1e3be7"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.318219 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal/instance:
uuid: "6acc2c81b8294930bc96491e466c1c87"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "9f5dcd387758899cfe22804ffee887f6"
server_key_iv: "34b495e735be83c980eb236c7f1e3be7"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.319837 20835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260619 12:08:46.320906 20849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.321213 20835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:46.321322 20835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal
uuid: "6acc2c81b8294930bc96491e466c1c87"
format_stamp: "Formatted at 2026-06-19 12:08:46 on dist-test-slave-44vj"
server_key: "9f5dcd387758899cfe22804ffee887f6"
server_key_iv: "34b495e735be83c980eb236c7f1e3be7"
server_key_version: "encryptionkey@0"
I20260619 12:08:46.321399 20835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:46.344173 20835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:46.344580 20835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:46.344722 20835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:46.345000 20835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:08:46.345371 20835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:08:46.345405 20835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.345443 20835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:08:46.345463 20835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:46.352432 20835 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.131:45895
I20260619 12:08:46.352573 20962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.131:45895 every 8 connection(s)
I20260619 12:08:46.352885 20835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/data/info.pb
I20260619 12:08:46.358814 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.353954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:38167 (local address 127.0.203.190:34455)
0619 12:08:46.354087 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.354090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.354308 (+ 218us) server_negotiation.cc:407] Connection header received
0619 12:08:46.354903 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.354904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.354917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.354970 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:46.355708 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.356250 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.357305 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.357514 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.357571 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.358526 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:46.358528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:46.358528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:46.358535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:46.358554 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:46.358558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:46.358602 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.358689 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.358720 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260619 12:08:46.359036 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.353872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:34455 (local address 127.0.203.131:38167)
0619 12:08:46.354161 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.354204 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.354792 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.355045 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.355049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.355059 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:08:46.355615 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.355620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.356354 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.356355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.357214 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.357217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.357593 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.357594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.357735 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.358340 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:08:46.358341 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:08:46.358375 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:08:46.358406 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:08:46.358612 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:08:46.358617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:08:46.358621 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.358683 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.358800 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
I20260619 12:08:46.359671 20963 heartbeater.cc:347] Connected to a master server at 127.0.203.190:34455
I20260619 12:08:46.359767 20963 heartbeater.cc:464] Registering TS with master...
I20260619 12:08:46.359946 20963 heartbeater.cc:511] Master 127.0.203.190:34455 requested a full tablet report, sending...
I20260619 12:08:46.360308 20513 ts_manager.cc:194] Registered new tserver with Master: 6acc2c81b8294930bc96491e466c1c87 (127.0.203.131:45895)
I20260619 12:08:46.360668 20513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.203.131:38167
I20260619 12:08:46.362717 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 20835
I20260619 12:08:46.362823 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal/instance
I20260619 12:08:46.363054 814 external_mini_cluster.cc:1428] Setting key b577e7125d72a3b6d408aa65d4c2addc
I20260619 12:08:46.364368 814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:08:46.369280 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.366008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55834 (local address 127.0.203.190:34455)
0619 12:08:46.366121 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.366123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.366199 (+ 76us) server_negotiation.cc:407] Connection header received
0619 12:08:46.366285 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.366286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.366299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.366350 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:46.366525 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.367078 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.367915 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.368135 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.368194 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.369014 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:46.369016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:46.369017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:46.369023 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:46.369040 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:46.369044 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:46.369089 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.369148 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.369177 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 12:08:46.371306 20513 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:08:46.373370 20513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:08:46.381501 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.375991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39509 (local address 127.0.0.1:47498)
0619 12:08:46.376589 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.376633 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.376724 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.377028 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.377031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.377034 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:46.377121 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.377123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.378530 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.378532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.380014 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.380017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.381147 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.381149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.381235 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.381252 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.381326 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.381398 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":83,"thread_start_us":69,"threads_started":1}
I20260619 12:08:46.381642 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.376279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.0.203.129:39509)
0619 12:08:46.376494 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.376522 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.376717 (+ 195us) server_negotiation.cc:407] Connection header received
0619 12:08:46.376827 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.376830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.376847 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.376931 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:46.377204 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.378429 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.380109 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.380919 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.380989 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.381106 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.381402 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.381487 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260619 12:08:46.381886 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.376277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.0.203.130:46879)
0619 12:08:46.376494 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.376500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.376827 (+ 327us) server_negotiation.cc:407] Connection header received
0619 12:08:46.376929 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.376933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.376954 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.377035 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:46.377321 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.378454 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.380208 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.380917 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.380989 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.381106 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.381735 (+ 629us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.381792 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260619 12:08:46.382176 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.375991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:46879 (local address 127.0.0.1:43796)
0619 12:08:46.376729 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.376744 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.376823 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.377130 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.377131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.377134 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:46.377233 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.377242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.378555 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.378555 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.380114 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.380116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.381437 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.381437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.381511 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.381515 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.382082 (+ 567us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.382127 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":47,"thread_start_us":113,"threads_started":1}
I20260619 12:08:46.382493 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.376012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.131:45895 (local address 127.0.0.1:46520)
0619 12:08:46.376924 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.376933 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.376986 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.378243 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.378244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.378246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:46.378297 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.378299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.379656 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.379657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.380894 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.380897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.382214 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.382215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.382287 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.382292 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.382345 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.382405 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"thread_start_us":132,"threads_started":1}
I20260619 12:08:46.382463 20635 tablet_service.cc:1467] Processing CreateTablet for tablet 6a41bb022ed941498433d99982a51844 (DEFAULT_TABLE table=test-table [id=e1cef6fa1ca24214ae7d573000718993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:46.382797 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.376327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.0.203.131:45895)
0619 12:08:46.376610 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.376614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.378073 (+ 1459us) server_negotiation.cc:407] Connection header received
0619 12:08:46.378130 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.378133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.378150 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.378235 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:46.378374 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.379555 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.380981 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.381585 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.381878 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.381982 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.382662 (+ 680us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.382722 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
I20260619 12:08:46.382853 20635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a41bb022ed941498433d99982a51844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:46.382925 20766 tablet_service.cc:1467] Processing CreateTablet for tablet 6a41bb022ed941498433d99982a51844 (DEFAULT_TABLE table=test-table [id=e1cef6fa1ca24214ae7d573000718993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:46.383311 20766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a41bb022ed941498433d99982a51844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:46.383558 20897 tablet_service.cc:1467] Processing CreateTablet for tablet 6a41bb022ed941498433d99982a51844 (DEFAULT_TABLE table=test-table [id=e1cef6fa1ca24214ae7d573000718993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:08:46.383886 20897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a41bb022ed941498433d99982a51844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:46.385938 20977 tablet_bootstrap.cc:492] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf: Bootstrap starting.
I20260619 12:08:46.386628 20977 tablet_bootstrap.cc:654] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:46.386931 20977 log.cc:824] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:46.387262 20978 tablet_bootstrap.cc:492] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Bootstrap starting.
I20260619 12:08:46.387527 20979 tablet_bootstrap.cc:492] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Bootstrap starting.
I20260619 12:08:46.387924 20977 tablet_bootstrap.cc:492] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf: No bootstrap required, opened a new log
I20260619 12:08:46.388010 20977 ts_tablet_manager.cc:1397] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:08:46.388051 20978 tablet_bootstrap.cc:654] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:46.388163 20979 tablet_bootstrap.cc:654] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:46.388417 20978 log.cc:824] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:46.388453 20979 log.cc:824] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:46.389230 20979 tablet_bootstrap.cc:492] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: No bootstrap required, opened a new log
I20260619 12:08:46.389310 20979 ts_tablet_manager.cc:1397] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:08:46.390393 20977 raft_consensus.cc:348] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.390554 20977 raft_consensus.cc:374] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:46.390588 20977 raft_consensus.cc:729] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f37d698cfbf40aeb6d0c684d2855ddf, State: Initialized, Role: FOLLOWER
I20260619 12:08:46.390718 20977 consensus_queue.cc:260] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.391009 20977 ts_tablet_manager.cc:1428] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:08:46.391108 20701 heartbeater.cc:503] Master 127.0.203.190:34455 was elected leader, sending a full tablet report...
I20260619 12:08:46.391645 20979 raft_consensus.cc:348] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.391764 20978 tablet_bootstrap.cc:492] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: No bootstrap required, opened a new log
I20260619 12:08:46.391804 20979 raft_consensus.cc:374] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:46.391834 20978 ts_tablet_manager.cc:1397] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 12:08:46.391839 20979 raft_consensus.cc:729] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6acc2c81b8294930bc96491e466c1c87, State: Initialized, Role: FOLLOWER
I20260619 12:08:46.391985 20979 consensus_queue.cc:260] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.392292 20979 ts_tablet_manager.cc:1428] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 12:08:46.392330 20963 heartbeater.cc:503] Master 127.0.203.190:34455 was elected leader, sending a full tablet report...
I20260619 12:08:46.394205 20978 raft_consensus.cc:348] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.394337 20978 raft_consensus.cc:374] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:46.394371 20978 raft_consensus.cc:729] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3b5d0d2c50478cb15b122c04b1dc47, State: Initialized, Role: FOLLOWER
I20260619 12:08:46.394484 20978 consensus_queue.cc:260] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.394759 20978 ts_tablet_manager.cc:1428] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:08:46.394819 20832 heartbeater.cc:503] Master 127.0.203.190:34455 was elected leader, sending a full tablet report...
W20260619 12:08:46.462205 20833 tablet.cc:2369] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:46.561034 20984 raft_consensus.cc:482] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:08:46.561213 20984 raft_consensus.cc:504] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.561690 20984 leader_election.cc:288] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3b5d0d2c50478cb15b122c04b1dc47 (127.0.203.130:46879), 6f37d698cfbf40aeb6d0c684d2855ddf (127.0.203.129:39509)
I20260619 12:08:46.564972 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.561884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.130:46879 (local address 127.0.203.131:52501)
0619 12:08:46.561971 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.561984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.562029 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.562238 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.562240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.562243 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:46.562335 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.562338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.563159 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.563162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.564099 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.564102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.564677 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.564678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.564751 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.564755 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.564800 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.564851 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 12:08:46.564967 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.561992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:52501 (local address 127.0.203.130:46879)
0619 12:08:46.562066 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.562069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.562078 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:08:46.562117 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.562118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.562130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.562192 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:46.562485 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.563063 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.564190 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.564601 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.564670 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.564681 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.564813 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.564864 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:08:46.565407 20786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a41bb022ed941498433d99982a51844" candidate_uuid: "6acc2c81b8294930bc96491e466c1c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" is_pre_election: true
I20260619 12:08:46.565629 20786 raft_consensus.cc:2454] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6acc2c81b8294930bc96491e466c1c87 in term 0.
I20260619 12:08:46.565905 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.561961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39509 (local address 127.0.203.131:52749)
0619 12:08:46.562183 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.562192 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:08:46.562254 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:08:46.563120 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:08:46.563121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:08:46.563123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:08:46.563205 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.563207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.563951 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.563952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.564806 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:08:46.564809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.565559 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.565560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:08:46.565657 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.565663 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:08:46.565776 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 12:08:46.565826 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":137,"threads_started":1}
I20260619 12:08:46.565927 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.562097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.203.131:52749 (local address 127.0.203.129:39509)
0619 12:08:46.562153 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.562156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.562262 (+ 106us) server_negotiation.cc:407] Connection header received
0619 12:08:46.562304 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.562305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.562318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.562416 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:08:46.563331 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.563877 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.564892 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.565424 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.565513 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.565524 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.565795 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.565849 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:08:46.565915 20852 leader_election.cc:302] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3b5d0d2c50478cb15b122c04b1dc47, 6acc2c81b8294930bc96491e466c1c87; no voters:
I20260619 12:08:46.566169 20984 raft_consensus.cc:2781] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:08:46.566272 20984 raft_consensus.cc:482] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:08:46.566293 20984 raft_consensus.cc:3037] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:46.566406 20655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a41bb022ed941498433d99982a51844" candidate_uuid: "6acc2c81b8294930bc96491e466c1c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" is_pre_election: true
I20260619 12:08:46.566597 20655 raft_consensus.cc:2454] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6acc2c81b8294930bc96491e466c1c87 in term 0.
I20260619 12:08:46.567220 20984 raft_consensus.cc:504] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.567427 20984 leader_election.cc:288] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [CANDIDATE]: Term 1 election: Requested vote from peers 2c3b5d0d2c50478cb15b122c04b1dc47 (127.0.203.130:46879), 6f37d698cfbf40aeb6d0c684d2855ddf (127.0.203.129:39509)
I20260619 12:08:46.567598 20655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a41bb022ed941498433d99982a51844" candidate_uuid: "6acc2c81b8294930bc96491e466c1c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf"
I20260619 12:08:46.567628 20786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a41bb022ed941498433d99982a51844" candidate_uuid: "6acc2c81b8294930bc96491e466c1c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47"
I20260619 12:08:46.567703 20655 raft_consensus.cc:3037] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:46.567715 20786 raft_consensus.cc:3037] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:46.568849 20655 raft_consensus.cc:2454] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6acc2c81b8294930bc96491e466c1c87 in term 1.
I20260619 12:08:46.568849 20786 raft_consensus.cc:2454] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6acc2c81b8294930bc96491e466c1c87 in term 1.
I20260619 12:08:46.569092 20852 leader_election.cc:302] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3b5d0d2c50478cb15b122c04b1dc47, 6acc2c81b8294930bc96491e466c1c87; no voters:
I20260619 12:08:46.569231 20984 raft_consensus.cc:2781] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:46.569463 20984 raft_consensus.cc:686] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 1 LEADER]: Becoming Leader. State: Replica: 6acc2c81b8294930bc96491e466c1c87, State: Running, Role: LEADER
I20260619 12:08:46.569572 20984 consensus_queue.cc:237] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } }
I20260619 12:08:46.570664 20512 catalog_manager.cc:5495] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6acc2c81b8294930bc96491e466c1c87 (127.0.203.131). New cstate: current_term: 1 leader_uuid: "6acc2c81b8294930bc96491e466c1c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6acc2c81b8294930bc96491e466c1c87" member_type: VOTER last_known_addr { host: "127.0.203.131" port: 45895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 } health_report { overall_health: UNKNOWN } } }
W20260619 12:08:46.580772 20702 tablet.cc:2369] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:08:46.603665 20964 tablet.cc:2369] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:08:46.609616 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.607087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.0.203.131:45895)
0619 12:08:46.607161 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:46.607163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:46.607174 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:46.607216 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:46.607217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:46.607240 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:46.607311 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:08:46.607448 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.607928 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.608754 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:46.608949 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:46.608998 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:46.609218 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:08:46.609325 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:08:46.609396 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:46.609498 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:08:46.609518 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:08:46.611701 20786 raft_consensus.cc:1261] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 1 FOLLOWER]: Refusing update from remote peer 6acc2c81b8294930bc96491e466c1c87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:46.611701 20655 raft_consensus.cc:1261] T 6a41bb022ed941498433d99982a51844 P 6f37d698cfbf40aeb6d0c684d2855ddf [term 1 FOLLOWER]: Refusing update from remote peer 6acc2c81b8294930bc96491e466c1c87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:08:46.612143 20987 consensus_queue.cc:1035] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3b5d0d2c50478cb15b122c04b1dc47" member_type: VOTER last_known_addr { host: "127.0.203.130" port: 46879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:46.612408 20984 consensus_queue.cc:1035] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f37d698cfbf40aeb6d0c684d2855ddf" member_type: VOTER last_known_addr { host: "127.0.203.129" port: 39509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:08:46.615334 20992 mvcc.cc:205] Tried to move back new op lower bound from 7298543315398176768 to 7298543315229806592. Current Snapshot: MvccSnapshot[applied={T|T < 7298543315398176768}]
I20260619 12:08:46.615947 20991 mvcc.cc:205] Tried to move back new op lower bound from 7298543315398176768 to 7298543315229806592. Current Snapshot: MvccSnapshot[applied={T|T < 7298543315398176768}]
I20260619 12:08:46.617630 20512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:46.617750 20512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:08:46.618469 20512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80abe9d83652477da0f773a124a932da: Sending DeleteTablet for 3 replicas of tablet 6a41bb022ed941498433d99982a51844
I20260619 12:08:46.618798 20635 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a41bb022ed941498433d99982a51844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:46 UTC) from {username='slave'} at 127.0.0.1:47498
I20260619 12:08:46.618835 20897 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a41bb022ed941498433d99982a51844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:46 UTC) from {username='slave'} at 127.0.0.1:46520
I20260619 12:08:46.618966 20766 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a41bb022ed941498433d99982a51844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:08:46 UTC) from {username='slave'} at 127.0.0.1:43796
I20260619 12:08:46.619259 21004 tablet_replica.cc:330] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: stopping tablet replica
I20260619 12:08:46.619405 21004 raft_consensus.cc:2229] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:08:46.619524 21004 raft_consensus.cc:2258] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:46.619757 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 20573
I20260619 12:08:46.619985 21004 ts_tablet_manager.cc:1905] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:46.621621 21004 ts_tablet_manager.cc:1918] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:46.621687 21004 log.cc:1196] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-1/wal/wals/6a41bb022ed941498433d99982a51844
I20260619 12:08:46.621925 21004 ts_tablet_manager.cc:1939] T 6a41bb022ed941498433d99982a51844 P 2c3b5d0d2c50478cb15b122c04b1dc47: Deleting consensus metadata
I20260619 12:08:46.622336 20499 catalog_manager.cc:4841] TS 2c3b5d0d2c50478cb15b122c04b1dc47 (127.0.203.130:46879): tablet 6a41bb022ed941498433d99982a51844 (table test-table [id=e1cef6fa1ca24214ae7d573000718993]) successfully deleted
I20260619 12:08:46.627578 21002 tablet_replica.cc:330] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: stopping tablet replica
W20260619 12:08:46.627553 20500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.203.129:39509 (error 108)
I20260619 12:08:46.627750 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 20704
I20260619 12:08:46.627915 21002 raft_consensus.cc:2229] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:08:46.628150 21002 raft_consensus.cc:2258] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:08:46.628614 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:46.628467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.129:39509 (local address 127.0.0.1:47512)
0619 12:08:46.628520 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 12:08:46.628561 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.129:39509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:08:46.628734 20500 catalog_manager.cc:4584] TS 6f37d698cfbf40aeb6d0c684d2855ddf (127.0.203.129:39509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a41bb022ed941498433d99982a51844: Network error: Client connection negotiation failed: client connection to 127.0.203.129:39509: connect: Connection refused (error 111)
I20260619 12:08:46.629233 21002 ts_tablet_manager.cc:1905] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:08:46.631054 21002 ts_tablet_manager.cc:1918] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:08:46.631130 21002 log.cc:1196] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Deleting WAL directory at /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870673623844-814-0/minicluster-data/ts-2/wal/wals/6a41bb022ed941498433d99982a51844
I20260619 12:08:46.631381 21002 ts_tablet_manager.cc:1939] T 6a41bb022ed941498433d99982a51844 P 6acc2c81b8294930bc96491e466c1c87: Deleting consensus metadata
I20260619 12:08:46.631891 20498 catalog_manager.cc:4841] TS 6acc2c81b8294930bc96491e466c1c87 (127.0.203.131:45895): tablet 6a41bb022ed941498433d99982a51844 (table test-table [id=e1cef6fa1ca24214ae7d573000718993]) successfully deleted
I20260619 12:08:46.633634 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 20835
I20260619 12:08:46.638304 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 20482
2026-06-19T12:08:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (891 ms)
[----------] 6 tests from EncryptionPolicyTest (8496 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:46 dist-test-slave-44vj krb5kdc[21008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:46 dist-test-slave-44vj krb5kdc[21008](info): set up 2 sockets
Jun 19 12:08:46 dist-test-slave-44vj krb5kdc[21008](info): commencing operation
krb5kdc: starting...
W20260619 12:08:47.595660 814 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:47 dist-test-slave-44vj krb5kdc[21008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:47.723920 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:47.815932 21021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:47.816205 21021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:47.816228 21021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:47.817730 21021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:47.817802 21021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:47.817816 21021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:47.817827 21021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:47.817838 21021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:47.819656 21021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34113
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:47.819922 21021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:47.820151 21021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:47.825574 21027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:47.825634 21026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:47.825583 21029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:47.826611 21021 server_base.cc:1034] running on GCE node
I20260619 12:08:47.826907 21021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:47.827277 21021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:47.828455 21021 hybrid_clock.cc:648] HybridClock initialized: now 1781870927828417 us; error 66 us; skew 500 ppm
Jun 19 12:08:47 dist-test-slave-44vj krb5kdc[21008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870927, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:47.831866 21021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:47.832502 21021 webserver.cc:460] Webserver started at http://127.0.0.1:35253/ using document root <none> and password file <none>
I20260619 12:08:47.832773 21021 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:47.832826 21021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:47.832963 21021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:47.834266 21021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "fb43cfc9f2da41e5b7569f1406ea080a"
format_stamp: "Formatted at 2026-06-19 12:08:47 on dist-test-slave-44vj"
server_key: "fc4eb2e42bdb2d2fc4a5c88646dbdf7c"
server_key_iv: "f135969e9d1540665b73924d03ca97ad"
server_key_version: "encryptionkey@0"
I20260619 12:08:47.834658 21021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "fb43cfc9f2da41e5b7569f1406ea080a"
format_stamp: "Formatted at 2026-06-19 12:08:47 on dist-test-slave-44vj"
server_key: "fc4eb2e42bdb2d2fc4a5c88646dbdf7c"
server_key_iv: "f135969e9d1540665b73924d03ca97ad"
server_key_version: "encryptionkey@0"
I20260619 12:08:47.836211 21021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:08:47.837321 21036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:47.837641 21021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:47.837760 21021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "fb43cfc9f2da41e5b7569f1406ea080a"
format_stamp: "Formatted at 2026-06-19 12:08:47 on dist-test-slave-44vj"
server_key: "fc4eb2e42bdb2d2fc4a5c88646dbdf7c"
server_key_iv: "f135969e9d1540665b73924d03ca97ad"
server_key_version: "encryptionkey@0"
I20260619 12:08:47.837852 21021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:47.852909 21021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:47.855324 21021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:47.855511 21021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:47.860342 21021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34113
I20260619 12:08:47.860409 21088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34113 every 8 connection(s)
I20260619 12:08:47.860780 21021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:47.861749 21089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:47.864198 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a: Bootstrap starting.
I20260619 12:08:47.864790 21089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:47.865031 21089 log.cc:824] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:47.865911 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a: No bootstrap required, opened a new log
I20260619 12:08:47.867918 21089 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260619 12:08:47.868101 21089 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:47.868124 21089 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb43cfc9f2da41e5b7569f1406ea080a, State: Initialized, Role: FOLLOWER
I20260619 12:08:47.868257 21089 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260619 12:08:47.868356 21089 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:47.868383 21089 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:47.868412 21089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:47.869002 21089 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260619 12:08:47.869095 21089 leader_election.cc:302] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb43cfc9f2da41e5b7569f1406ea080a; no voters:
I20260619 12:08:47.869246 21089 leader_election.cc:288] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:47.869318 21092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:47.869529 21092 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [term 1 LEADER]: Becoming Leader. State: Replica: fb43cfc9f2da41e5b7569f1406ea080a, State: Running, Role: LEADER
I20260619 12:08:47.869562 21089 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:47.869647 21092 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260619 12:08:47.870190 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21021
I20260619 12:08:47.870299 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:47.870445 21094 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb43cfc9f2da41e5b7569f1406ea080a. Latest consensus state: current_term: 1 leader_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } } }
I20260619 12:08:47.870535 21094 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:47.870545 814 external_mini_cluster.cc:1428] Setting key d66498ce01f10705ee8fe2ac6cf1f556
I20260619 12:08:47.870828 21093 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb43cfc9f2da41e5b7569f1406ea080a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } } }
I20260619 12:08:47.870982 21093 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:47.871033 21101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:47.871754 21101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:47.873665 21101 catalog_manager.cc:1347] Generated new cluster ID: b05f86ba72eb4218a14301fe77074a43
I20260619 12:08:47.873726 21101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:08:47 dist-test-slave-44vj krb5kdc[21008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:08:47.881738 21101 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:47.882377 21101 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:47.884186 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:47.871299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57395 (local address 127.0.0.1:34113)
0619 12:08:47.871589 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:47.871601 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:47.871638 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:08:47.871767 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:47.871774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:47.871833 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:47.871978 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:47.872748 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:47.874194 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:47.875473 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:47.876156 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:47.876261 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:47.879521 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:47.879529 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:47.879530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:47.879550 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:47.881940 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:47.882526 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:47.882531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:47.882531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:47.882672 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:47.883010 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:47.883011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:47.883012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:47.883304 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:47.883540 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:47.883669 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 12:08:47.883911 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":129,"threads_started":1}
I20260619 12:08:47.885073 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:47.888882 21101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb43cfc9f2da41e5b7569f1406ea080a: Generated new TSK 0
I20260619 12:08:47.889082 21101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:47.893654 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:47.886755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47363 (local address 127.0.0.1:34113)
0619 12:08:47.886789 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:47.886790 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:47.887101 (+ 311us) server_negotiation.cc:407] Connection header received
0619 12:08:47.887171 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:47.887172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:47.887183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:47.887239 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:47.888036 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:47.888696 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:47.889688 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:47.889927 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:47.889991 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:47.891217 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:47.891219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:47.891219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:47.891227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:47.892548 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:47.892902 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:47.892904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:47.892904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:47.892969 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:47.893159 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:47.893159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:47.893160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:47.893279 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:47.893381 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:47.893498 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:08:47.893560 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":14}
I20260619 12:08:47.896018 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21021
W20260619 12:08:47.900244 21006 connection.cc:535] client connection to 127.0.0.1:34113 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34113): Connection reset by peer (error 104)
2026-06-19T12:08:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:47 dist-test-slave-44vj krb5kdc[21120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:47 dist-test-slave-44vj krb5kdc[21120](info): set up 2 sockets
Jun 19 12:08:47 dist-test-slave-44vj krb5kdc[21120](info): commencing operation
krb5kdc: starting...
W20260619 12:08:48.860178 814 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:48 dist-test-slave-44vj krb5kdc[21120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:48.998950 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:49.088117 21133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:49.088416 21133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:49.088455 21133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:49.089879 21133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:49.089927 21133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:49.089947 21133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:49.089964 21133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:49.089980 21133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:49.091815 21133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36353
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:49.092126 21133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:49.092377 21133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:49.097788 21138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:49.097961 21133 server_base.cc:1034] running on GCE node
W20260619 12:08:49.097843 21139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:49.097788 21141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:49.098388 21133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:49.098786 21133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:49.099948 21133 hybrid_clock.cc:648] HybridClock initialized: now 1781870929099919 us; error 48 us; skew 500 ppm
Jun 19 12:08:49 dist-test-slave-44vj krb5kdc[21120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870929, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:49.102972 21133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:49.103641 21133 webserver.cc:460] Webserver started at http://127.0.0.1:44301/ using document root <none> and password file <none>
I20260619 12:08:49.103893 21133 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:49.103950 21133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:49.104096 21133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:49.105469 21133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "a8f414a7d95a4e80942591d90b3c1f8f"
format_stamp: "Formatted at 2026-06-19 12:08:49 on dist-test-slave-44vj"
server_key: "42203eb51b5d3e4a3545e711e2f4479c"
server_key_iv: "bd9db0e9c1351b450dbd415007296278"
server_key_version: "encryptionkey@0"
I20260619 12:08:49.105890 21133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "a8f414a7d95a4e80942591d90b3c1f8f"
format_stamp: "Formatted at 2026-06-19 12:08:49 on dist-test-slave-44vj"
server_key: "42203eb51b5d3e4a3545e711e2f4479c"
server_key_iv: "bd9db0e9c1351b450dbd415007296278"
server_key_version: "encryptionkey@0"
I20260619 12:08:49.107543 21133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260619 12:08:49.108508 21148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:49.108778 21133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:49.108882 21133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "a8f414a7d95a4e80942591d90b3c1f8f"
format_stamp: "Formatted at 2026-06-19 12:08:49 on dist-test-slave-44vj"
server_key: "42203eb51b5d3e4a3545e711e2f4479c"
server_key_iv: "bd9db0e9c1351b450dbd415007296278"
server_key_version: "encryptionkey@0"
I20260619 12:08:49.108975 21133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:49.124785 21133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:49.127202 21133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:49.127415 21133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:49.132251 21133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36353
I20260619 12:08:49.132316 21200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36353 every 8 connection(s)
I20260619 12:08:49.132704 21133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:49.133715 21201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:49.135195 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21133
I20260619 12:08:49.135341 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:49.135564 814 external_mini_cluster.cc:1428] Setting key 680a149f317714601f6fcd3bc8de6db6
I20260619 12:08:49.136646 21201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f: Bootstrap starting.
I20260619 12:08:49.137305 21201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:49.137621 21201 log.cc:824] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:49.138350 21201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f: No bootstrap required, opened a new log
I20260619 12:08:49.140681 21201 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260619 12:08:49.140920 21201 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:49.140957 21201 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8f414a7d95a4e80942591d90b3c1f8f, State: Initialized, Role: FOLLOWER
I20260619 12:08:49.141127 21201 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260619 12:08:49.141199 21201 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:49.141230 21201 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:49.141269 21201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:49.142025 21201 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260619 12:08:49.142145 21201 leader_election.cc:302] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8f414a7d95a4e80942591d90b3c1f8f; no voters:
I20260619 12:08:49.142346 21201 leader_election.cc:288] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:49.142388 21206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:49.142413 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:49.136282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33557 (local address 127.0.0.1:36353)
0619 12:08:49.136489 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:49.136499 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:49.136534 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:08:49.137199 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:49.137209 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:49.137271 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:49.137449 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:49.137781 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:49.138882 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:49.140067 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:49.140578 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:49.140688 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:49.141587 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:49.141592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:49.141592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:49.141807 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:49.141851 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:49.141864 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:49.141936 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:49.142114 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 12:08:49.142156 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260619 12:08:49.142508 21206 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [term 1 LEADER]: Becoming Leader. State: Replica: a8f414a7d95a4e80942591d90b3c1f8f, State: Running, Role: LEADER
I20260619 12:08:49.142658 21201 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:49.142808 21206 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260619 12:08:49.143350 21208 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8f414a7d95a4e80942591d90b3c1f8f. Latest consensus state: current_term: 1 leader_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } } }
I20260619 12:08:49.143493 21207 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f414a7d95a4e80942591d90b3c1f8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } } }
I20260619 12:08:49.143546 21208 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:49.143622 21207 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:49.143805 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:49.143872 21215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:49.144480 21215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:49.147488 21215 catalog_manager.cc:1347] Generated new cluster ID: 8c8710e7b421451eacb9209dd4037bd2
I20260619 12:08:49.147567 21215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:49.148939 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:49.145627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47279 (local address 127.0.0.1:36353)
0619 12:08:49.145671 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:49.145674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:49.145797 (+ 123us) server_negotiation.cc:407] Connection header received
0619 12:08:49.145855 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:49.145857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:49.145869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:49.145918 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:49.146062 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:49.146677 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:49.147490 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:49.147799 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:49.147842 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:49.148685 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:49.148687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:49.148687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:49.148692 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:49.148710 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:49.148715 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:49.148743 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:49.148846 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:08:49.148858 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:08:49.157979 21215 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:49.158721 21215 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:49.167554 21215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8f414a7d95a4e80942591d90b3c1f8f: Generated new TSK 0
I20260619 12:08:49.167783 21215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:49.179647 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21133
W20260619 12:08:49.184782 21118 connection.cc:535] client connection to 127.0.0.1:36353 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36353): Connection reset by peer (error 104)
2026-06-19T12:08:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:49 dist-test-slave-44vj krb5kdc[21232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:49 dist-test-slave-44vj krb5kdc[21232](info): set up 2 sockets
Jun 19 12:08:49 dist-test-slave-44vj krb5kdc[21232](info): commencing operation
krb5kdc: starting...
W20260619 12:08:50.143492 814 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:50 dist-test-slave-44vj krb5kdc[21232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:50.262771 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:50.350495 21245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:50.350745 21245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:50.350777 21245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:50.352028 21245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:50.352075 21245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:50.352093 21245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:50.352109 21245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:50.352123 21245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:50.353662 21245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:50.353935 21245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:50.354161 21245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:50.359419 21253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:50.359480 21251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:50.359433 21250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:50.359705 21245 server_base.cc:1034] running on GCE node
I20260619 12:08:50.360008 21245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:50.360342 21245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:50.361522 21245 hybrid_clock.cc:648] HybridClock initialized: now 1781870930361494 us; error 44 us; skew 500 ppm
Jun 19 12:08:50 dist-test-slave-44vj krb5kdc[21232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870930, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:50.364378 21245 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:50.364903 21245 webserver.cc:460] Webserver started at http://127.0.0.1:41287/ using document root <none> and password file <none>
I20260619 12:08:50.365125 21245 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:50.365175 21245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:50.365307 21245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:50.366429 21245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "1c600205f0e64595815b846933060fca"
format_stamp: "Formatted at 2026-06-19 12:08:50 on dist-test-slave-44vj"
server_key: "2beadc7a15d83f19a8e4e5640c71b752"
server_key_iv: "05852e86a555df245a7bb14bdffb7df3"
server_key_version: "encryptionkey@0"
I20260619 12:08:50.366787 21245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "1c600205f0e64595815b846933060fca"
format_stamp: "Formatted at 2026-06-19 12:08:50 on dist-test-slave-44vj"
server_key: "2beadc7a15d83f19a8e4e5640c71b752"
server_key_iv: "05852e86a555df245a7bb14bdffb7df3"
server_key_version: "encryptionkey@0"
I20260619 12:08:50.368294 21245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:50.369161 21260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:50.369407 21245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:50.369514 21245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "1c600205f0e64595815b846933060fca"
format_stamp: "Formatted at 2026-06-19 12:08:50 on dist-test-slave-44vj"
server_key: "2beadc7a15d83f19a8e4e5640c71b752"
server_key_iv: "05852e86a555df245a7bb14bdffb7df3"
server_key_version: "encryptionkey@0"
I20260619 12:08:50.369603 21245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:50.373256 21245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:50.375308 21245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:50.375488 21245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:50.379689 21245 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36661
I20260619 12:08:50.379741 21312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36661 every 8 connection(s)
I20260619 12:08:50.380111 21245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:50.381212 21313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:50.383888 21313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca: Bootstrap starting.
I20260619 12:08:50.384521 21313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:50.384806 21313 log.cc:824] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:50.385686 21313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca: No bootstrap required, opened a new log
I20260619 12:08:50.387480 21313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c600205f0e64595815b846933060fca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260619 12:08:50.387636 21313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:50.387668 21313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c600205f0e64595815b846933060fca, State: Initialized, Role: FOLLOWER
I20260619 12:08:50.387820 21313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c600205f0e64595815b846933060fca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260619 12:08:50.387887 21313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:50.387918 21313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:50.387966 21313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:50.388542 21313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c600205f0e64595815b846933060fca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260619 12:08:50.388581 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21245
I20260619 12:08:50.388643 21313 leader_election.cc:302] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c600205f0e64595815b846933060fca; no voters:
I20260619 12:08:50.388681 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:50.388829 21313 leader_election.cc:288] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:50.388850 814 external_mini_cluster.cc:1428] Setting key 01c0f6503ff2153382cecf4e265b9d78
I20260619 12:08:50.388878 21316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:50.389154 21313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:50.389228 21316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [term 1 LEADER]: Becoming Leader. State: Replica: 1c600205f0e64595815b846933060fca, State: Running, Role: LEADER
I20260619 12:08:50.389434 21316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c600205f0e64595815b846933060fca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260619 12:08:50.389952 21318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c600205f0e64595815b846933060fca. Latest consensus state: current_term: 1 leader_uuid: "1c600205f0e64595815b846933060fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c600205f0e64595815b846933060fca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } } }
I20260619 12:08:50.390041 21318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:50.390256 21317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c600205f0e64595815b846933060fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c600205f0e64595815b846933060fca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } } }
I20260619 12:08:50.390383 21317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:50.391050 21327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:50.391656 21327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:50.392957 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:50.389359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42261 (local address 127.0.0.1:36661)
0619 12:08:50.389667 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:50.389680 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:50.389718 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:08:50.389882 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:50.389890 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:50.389927 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:50.390377 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:50.392405 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:50.392409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:50.392409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:50.392415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:50.392461 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:50.392476 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:50.392573 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:50.392695 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 12:08:50.392743 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":84,"threads_started":1}
I20260619 12:08:50.393404 21327 catalog_manager.cc:1347] Generated new cluster ID: fc6d58548cd345d299d611a337f7a1b3
I20260619 12:08:50.393455 21327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:50.394055 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:50.399762 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:50.395633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34591 (local address 127.0.0.1:36661)
0619 12:08:50.395678 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:50.395680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:50.398460 (+ 2780us) server_negotiation.cc:407] Connection header received
0619 12:08:50.398746 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:50.398748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:50.398760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:50.398843 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:50.399494 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:50.399496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:50.399496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:50.399501 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:50.399519 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:50.399524 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:50.399575 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:50.399634 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:08:50.399654 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:08:50.440455 21327 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:50.440712 21327 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:50.443315 21327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c600205f0e64595815b846933060fca: Generated new TSK 0
I20260619 12:08:50.443552 21327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:50.457175 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21245
2026-06-19T12:08:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:50 dist-test-slave-44vj krb5kdc[21344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:50 dist-test-slave-44vj krb5kdc[21344](info): set up 2 sockets
Jun 19 12:08:50 dist-test-slave-44vj krb5kdc[21344](info): commencing operation
krb5kdc: starting...
W20260619 12:08:51.423684 814 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:51 dist-test-slave-44vj krb5kdc[21344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:51.547994 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36197
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:51.637394 21357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:51.637650 21357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:51.637682 21357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:51.638883 21357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:51.638926 21357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:51.638944 21357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:51.638960 21357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:51.638984 21357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:51.640559 21357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:51.640831 21357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:51.641129 21357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:51.646329 21362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:51.646327 21365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:51.646347 21363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:51.646636 21357 server_base.cc:1034] running on GCE node
I20260619 12:08:51.646958 21357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:51.647290 21357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:51.648469 21357 hybrid_clock.cc:648] HybridClock initialized: now 1781870931648450 us; error 34 us; skew 500 ppm
Jun 19 12:08:51 dist-test-slave-44vj krb5kdc[21344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870931, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:51.651309 21357 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:51.651868 21357 webserver.cc:460] Webserver started at http://127.0.0.1:45261/ using document root <none> and password file <none>
I20260619 12:08:51.652091 21357 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:51.652143 21357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:51.652273 21357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:51.653415 21357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "5b0ee62692844bb9852c314af94506c4"
format_stamp: "Formatted at 2026-06-19 12:08:51 on dist-test-slave-44vj"
server_key: "fb9a3cda632fc6623ce64f46103cde28"
server_key_iv: "de304834b693d9a85f591f51f806f629"
server_key_version: "encryptionkey@0"
I20260619 12:08:51.653808 21357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "5b0ee62692844bb9852c314af94506c4"
format_stamp: "Formatted at 2026-06-19 12:08:51 on dist-test-slave-44vj"
server_key: "fb9a3cda632fc6623ce64f46103cde28"
server_key_iv: "de304834b693d9a85f591f51f806f629"
server_key_version: "encryptionkey@0"
I20260619 12:08:51.655215 21357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 12:08:51.656087 21372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:51.656342 21357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:51.656454 21357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "5b0ee62692844bb9852c314af94506c4"
format_stamp: "Formatted at 2026-06-19 12:08:51 on dist-test-slave-44vj"
server_key: "fb9a3cda632fc6623ce64f46103cde28"
server_key_iv: "de304834b693d9a85f591f51f806f629"
server_key_version: "encryptionkey@0"
I20260619 12:08:51.656536 21357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:51.674584 21357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:51.677024 21357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:51.677163 21357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:51.681644 21357 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36197
I20260619 12:08:51.681704 21424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36197 every 8 connection(s)
I20260619 12:08:51.682068 21357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:51.683159 21425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:51.684100 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21357
I20260619 12:08:51.684216 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:51.684409 814 external_mini_cluster.cc:1428] Setting key d1b016f04905ec4816cc656c3a16f402
I20260619 12:08:51.686713 21425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4: Bootstrap starting.
I20260619 12:08:51.687598 21425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:51.688016 21425 log.cc:824] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:51.688969 21425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4: No bootstrap required, opened a new log
I20260619 12:08:51.691099 21425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0ee62692844bb9852c314af94506c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260619 12:08:51.691288 21425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:51.691322 21425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0ee62692844bb9852c314af94506c4, State: Initialized, Role: FOLLOWER
I20260619 12:08:51.691475 21425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0ee62692844bb9852c314af94506c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260619 12:08:51.691553 21425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:51.691586 21425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:51.691633 21425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 12:08:51 dist-test-slave-44vj krb5kdc[21344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:08:51.692384 21425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0ee62692844bb9852c314af94506c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260619 12:08:51.692489 21425 leader_election.cc:302] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b0ee62692844bb9852c314af94506c4; no voters:
I20260619 12:08:51.692682 21425 leader_election.cc:288] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:51.692754 21430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:51.692951 21430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [term 1 LEADER]: Becoming Leader. State: Replica: 5b0ee62692844bb9852c314af94506c4, State: Running, Role: LEADER
I20260619 12:08:51.693024 21425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:51.693153 21430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0ee62692844bb9852c314af94506c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260619 12:08:51.693768 21432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b0ee62692844bb9852c314af94506c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0ee62692844bb9852c314af94506c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } } }
I20260619 12:08:51.693899 21432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:51.694140 21439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:51.693751 21431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b0ee62692844bb9852c314af94506c4. Latest consensus state: current_term: 1 leader_uuid: "5b0ee62692844bb9852c314af94506c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0ee62692844bb9852c314af94506c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } } }
I20260619 12:08:51.694458 21431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:51.694813 21439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:51.696604 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:51.685065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56601 (local address 127.0.0.1:36197)
0619 12:08:51.685312 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:51.685324 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:51.685369 (+ 45us) server_negotiation.cc:407] Connection header received
0619 12:08:51.685894 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:51.685904 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:51.685976 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:51.686148 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:51.686908 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:51.688429 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:51.689286 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:51.689738 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:51.689849 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:51.692409 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:51.692414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:51.692414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:51.692429 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:51.694689 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:51.695228 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:51.695232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:51.695232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:51.695342 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:51.695594 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:51.695597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:51.695597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:51.695950 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:51.696113 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:51.696227 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 12:08:51.696405 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":126,"threads_started":1}
I20260619 12:08:51.696990 21439 catalog_manager.cc:1347] Generated new cluster ID: d4dbc9eed6744fbaaeb440d3ae3a7c66
I20260619 12:08:51.697050 21439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:51.697649 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:51.705947 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:51.699060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35319 (local address 127.0.0.1:36197)
0619 12:08:51.699095 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:51.699097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:51.699229 (+ 132us) server_negotiation.cc:407] Connection header received
0619 12:08:51.699289 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:51.699289 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:51.699299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:51.699347 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:51.699886 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:51.700453 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:51.701490 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:51.701729 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:51.701789 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:51.702977 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:51.702979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:51.702979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:51.702986 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:51.704619 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:51.705087 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:51.705088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:51.705088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:51.705154 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:51.705365 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:51.705365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:51.705366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:51.705508 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:51.705602 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:51.705746 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 12:08:51.705842 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 12:08:51.718011 21439 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:51.718598 21439 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:51.727182 21439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b0ee62692844bb9852c314af94506c4: Generated new TSK 0
I20260619 12:08:51.727528 21439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:51.730527 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21357
2026-06-19T12:08:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:51 dist-test-slave-44vj krb5kdc[21456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:51 dist-test-slave-44vj krb5kdc[21456](info): set up 2 sockets
Jun 19 12:08:51 dist-test-slave-44vj krb5kdc[21456](info): commencing operation
krb5kdc: starting...
W20260619 12:08:52.695739 814 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:52 dist-test-slave-44vj krb5kdc[21456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:52.824756 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:52.913439 21469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:52.913710 21469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:52.913738 21469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:52.914983 21469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:52.915030 21469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:52.915042 21469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:52.915053 21469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:52.915064 21469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:52.916667 21469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:52.916917 21469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:52.917130 21469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:52.922573 21475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:52.922730 21469 server_base.cc:1034] running on GCE node
W20260619 12:08:52.922545 21477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:52.922573 21474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:52.923326 21469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:52.923691 21469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:52.924880 21469 hybrid_clock.cc:648] HybridClock initialized: now 1781870932924832 us; error 61 us; skew 500 ppm
Jun 19 12:08:52 dist-test-slave-44vj krb5kdc[21456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870932, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:52.927691 21469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:52.928220 21469 webserver.cc:460] Webserver started at http://127.0.0.1:39939/ using document root <none> and password file <none>
I20260619 12:08:52.928462 21469 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:52.928515 21469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:52.928651 21469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:52.929770 21469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "879e3592fa484bb99fc7c09710826463"
format_stamp: "Formatted at 2026-06-19 12:08:52 on dist-test-slave-44vj"
server_key: "d3393d0dbba2880b6a56b278a250ddea"
server_key_iv: "053ac14a2046d3a4ace01565f17f321f"
server_key_version: "encryptionkey@0"
I20260619 12:08:52.930151 21469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "879e3592fa484bb99fc7c09710826463"
format_stamp: "Formatted at 2026-06-19 12:08:52 on dist-test-slave-44vj"
server_key: "d3393d0dbba2880b6a56b278a250ddea"
server_key_iv: "053ac14a2046d3a4ace01565f17f321f"
server_key_version: "encryptionkey@0"
I20260619 12:08:52.931627 21469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:08:52.932449 21484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:52.932658 21469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:52.932754 21469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "879e3592fa484bb99fc7c09710826463"
format_stamp: "Formatted at 2026-06-19 12:08:52 on dist-test-slave-44vj"
server_key: "d3393d0dbba2880b6a56b278a250ddea"
server_key_iv: "053ac14a2046d3a4ace01565f17f321f"
server_key_version: "encryptionkey@0"
I20260619 12:08:52.932822 21469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:52.954766 21469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:52.957089 21469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:52.957247 21469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:52.961612 21469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42431
I20260619 12:08:52.961676 21536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42431 every 8 connection(s)
I20260619 12:08:52.962044 21469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:52.963089 21537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:52.965984 21537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463: Bootstrap starting.
I20260619 12:08:52.966673 21537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:52.966945 21537 log.cc:824] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:52.967900 21537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463: No bootstrap required, opened a new log
I20260619 12:08:52.969740 21537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e3592fa484bb99fc7c09710826463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260619 12:08:52.969919 21537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:52.969944 21537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879e3592fa484bb99fc7c09710826463, State: Initialized, Role: FOLLOWER
I20260619 12:08:52.970085 21537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e3592fa484bb99fc7c09710826463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260619 12:08:52.970168 21537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:52.970196 21537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:52.970227 21537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:52.970891 21537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e3592fa484bb99fc7c09710826463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260619 12:08:52.970918 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21469
I20260619 12:08:52.971042 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:52.971031 21537 leader_election.cc:302] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 879e3592fa484bb99fc7c09710826463; no voters:
I20260619 12:08:52.971254 814 external_mini_cluster.cc:1428] Setting key f91317279188a221407c9852887af7c0
I20260619 12:08:52.971261 21537 leader_election.cc:288] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:52.971462 21540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:52.971728 21540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [term 1 LEADER]: Becoming Leader. State: Replica: 879e3592fa484bb99fc7c09710826463, State: Running, Role: LEADER
I20260619 12:08:52.971863 21540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e3592fa484bb99fc7c09710826463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260619 12:08:52.972096 21537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:52.972317 21540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879e3592fa484bb99fc7c09710826463. Latest consensus state: current_term: 1 leader_uuid: "879e3592fa484bb99fc7c09710826463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e3592fa484bb99fc7c09710826463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } } }
I20260619 12:08:52.972486 21541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "879e3592fa484bb99fc7c09710826463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e3592fa484bb99fc7c09710826463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } } }
I20260619 12:08:52.972548 21541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:52.972466 21540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:52.973006 21548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:52.973694 21548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:52.975730 21548 catalog_manager.cc:1347] Generated new cluster ID: 3170018a2fe34fd8a4cd9e002812c5a0
I20260619 12:08:52.975800 21548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:52.977720 21544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:52.971863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45573 (local address 127.0.0.1:42431)
0619 12:08:52.972097 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:52.972109 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:52.972336 (+ 227us) server_negotiation.cc:407] Connection header received
0619 12:08:52.972574 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:52.972584 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:52.972660 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:52.972803 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:52.973258 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:52.974607 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:52.975546 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:52.976116 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:52.976221 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:52.976970 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:52.976976 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:52.976976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:52.977199 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:52.977240 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:52.977253 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:52.977310 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:52.977397 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 12:08:52.977477 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260619 12:08:52.978679 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:52.983541 21544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:52.980176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51751 (local address 127.0.0.1:42431)
0619 12:08:52.980218 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:52.980219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:52.980367 (+ 148us) server_negotiation.cc:407] Connection header received
0619 12:08:52.980417 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:52.980418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:52.980430 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:52.980476 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:52.980606 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:52.981403 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:52.982098 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:52.982326 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:52.982383 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:52.983220 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:52.983222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:52.983222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:52.983227 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:52.983246 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:52.983250 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:52.983295 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:52.983390 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 12:08:52.983444 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:08:52.993268 21548 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:52.994050 21548 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:53.000448 21548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 879e3592fa484bb99fc7c09710826463: Generated new TSK 0
I20260619 12:08:53.000689 21548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:53.003451 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21469
2026-06-19T12:08:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:53 dist-test-slave-44vj krb5kdc[21568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:53 dist-test-slave-44vj krb5kdc[21568](info): set up 2 sockets
Jun 19 12:08:53 dist-test-slave-44vj krb5kdc[21568](info): commencing operation
krb5kdc: starting...
W20260619 12:08:53.975750 814 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:54 dist-test-slave-44vj krb5kdc[21568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:54.106316 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:54.190644 21581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:54.190873 21581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:54.190908 21581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:54.192214 21581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:54.192276 21581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:54.192296 21581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:54.192312 21581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:54.192327 21581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:54.193902 21581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:54.194175 21581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:54.194406 21581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:54.199534 21587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:54.199534 21586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:54.199570 21589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:54.199571 21581 server_base.cc:1034] running on GCE node
I20260619 12:08:54.200129 21581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:54.200474 21581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:54.201625 21581 hybrid_clock.cc:648] HybridClock initialized: now 1781870934201599 us; error 50 us; skew 500 ppm
Jun 19 12:08:54 dist-test-slave-44vj krb5kdc[21568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870934, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:54.207602 21581 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:54.208092 21581 webserver.cc:460] Webserver started at http://127.0.0.1:37439/ using document root <none> and password file <none>
I20260619 12:08:54.208324 21581 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:54.208374 21581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:54.208493 21581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:54.209519 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "7f331e2d8c314ef99c5635e44bfa21b6"
format_stamp: "Formatted at 2026-06-19 12:08:54 on dist-test-slave-44vj"
server_key: "c347abd9b609cde9b7e389abd52061c8"
server_key_iv: "9761a10b08c11eacd89f79182d4143c7"
server_key_version: "encryptionkey@0"
I20260619 12:08:54.209839 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "7f331e2d8c314ef99c5635e44bfa21b6"
format_stamp: "Formatted at 2026-06-19 12:08:54 on dist-test-slave-44vj"
server_key: "c347abd9b609cde9b7e389abd52061c8"
server_key_iv: "9761a10b08c11eacd89f79182d4143c7"
server_key_version: "encryptionkey@0"
I20260619 12:08:54.211180 21581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 12:08:54.212128 21596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:54.212357 21581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:54.212456 21581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "7f331e2d8c314ef99c5635e44bfa21b6"
format_stamp: "Formatted at 2026-06-19 12:08:54 on dist-test-slave-44vj"
server_key: "c347abd9b609cde9b7e389abd52061c8"
server_key_iv: "9761a10b08c11eacd89f79182d4143c7"
server_key_version: "encryptionkey@0"
I20260619 12:08:54.212523 21581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:54.215903 21581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:54.218058 21581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:54.218211 21581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:54.222558 21581 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46459
I20260619 12:08:54.222673 21648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46459 every 8 connection(s)
I20260619 12:08:54.222945 21581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:54.224196 21649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:54.226902 21649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6: Bootstrap starting.
I20260619 12:08:54.227589 21649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:54.227886 21649 log.cc:824] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:54.228907 21649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6: No bootstrap required, opened a new log
I20260619 12:08:54.230686 21649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260619 12:08:54.230861 21649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:54.230885 21649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f331e2d8c314ef99c5635e44bfa21b6, State: Initialized, Role: FOLLOWER
I20260619 12:08:54.231102 21649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260619 12:08:54.231217 21649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:54.231243 21649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:54.231277 21649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:54.231948 21649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260619 12:08:54.232074 21649 leader_election.cc:302] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f331e2d8c314ef99c5635e44bfa21b6; no voters:
I20260619 12:08:54.232141 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21581
I20260619 12:08:54.232261 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:54.232287 21649 leader_election.cc:288] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:54.232404 21652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:54.232476 814 external_mini_cluster.cc:1428] Setting key e96d81f39c23e7c39dc9a381ff0a4be2
I20260619 12:08:54.232628 21649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:54.232658 21652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [term 1 LEADER]: Becoming Leader. State: Replica: 7f331e2d8c314ef99c5635e44bfa21b6, State: Running, Role: LEADER
I20260619 12:08:54.232788 21652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260619 12:08:54.233291 21653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } } }
I20260619 12:08:54.233460 21654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f331e2d8c314ef99c5635e44bfa21b6. Latest consensus state: current_term: 1 leader_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f331e2d8c314ef99c5635e44bfa21b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } } }
I20260619 12:08:54.233575 21654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:54.233536 21653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:54.233850 21663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:54.234440 21663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:54.236434 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:54.233206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56467 (local address 127.0.0.1:46459)
0619 12:08:54.233427 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:54.233438 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:54.233876 (+ 438us) server_negotiation.cc:407] Connection header received
0619 12:08:54.234059 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:54.234068 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:54.234108 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:54.234267 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:54.235801 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:54.235805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:54.235806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:54.235810 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:54.235881 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:54.235900 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:54.235969 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:54.236162 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 12:08:54.236216 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260619 12:08:54.236712 21663 catalog_manager.cc:1347] Generated new cluster ID: edc0ca8693bb40249f6650b6bd3eab21
I20260619 12:08:54.236793 21663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:54.237517 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:54.244642 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:54.239736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40613 (local address 127.0.0.1:46459)
0619 12:08:54.239775 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:54.239777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:54.243594 (+ 3817us) server_negotiation.cc:407] Connection header received
0619 12:08:54.243665 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:54.243666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:54.243675 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:54.243721 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:54.244363 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:54.244365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:54.244365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:54.244370 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:54.244388 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:54.244392 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:54.244444 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:54.244501 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:08:54.244519 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:08:54.289042 21663 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:54.289238 21663 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:54.300634 21663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f331e2d8c314ef99c5635e44bfa21b6: Generated new TSK 0
I20260619 12:08:54.300874 21663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:54.309968 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21581
2026-06-19T12:08:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:54 dist-test-slave-44vj krb5kdc[21680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:54 dist-test-slave-44vj krb5kdc[21680](info): set up 2 sockets
Jun 19 12:08:54 dist-test-slave-44vj krb5kdc[21680](info): commencing operation
krb5kdc: starting...
W20260619 12:08:55.271802 814 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:55 dist-test-slave-44vj krb5kdc[21680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:55.396440 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43909
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:55.485996 21693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:55.486268 21693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:55.486304 21693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:55.487645 21693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:55.487689 21693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:55.487709 21693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:55.487726 21693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:55.487743 21693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:55.489388 21693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:55.489655 21693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:55.489897 21693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:55.495489 21698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:55.495576 21699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:55.495500 21701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:55.495798 21693 server_base.cc:1034] running on GCE node
I20260619 12:08:55.496104 21693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:55.496461 21693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:55.497653 21693 hybrid_clock.cc:648] HybridClock initialized: now 1781870935497637 us; error 50 us; skew 500 ppm
Jun 19 12:08:55 dist-test-slave-44vj krb5kdc[21680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870935, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:55.501085 21693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:55.501648 21693 webserver.cc:460] Webserver started at http://127.0.0.1:39339/ using document root <none> and password file <none>
I20260619 12:08:55.501909 21693 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:55.501977 21693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:55.502211 21693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:55.503331 21693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4"
format_stamp: "Formatted at 2026-06-19 12:08:55 on dist-test-slave-44vj"
server_key: "494d12ba3504d3b7c7714ff7ac964d82"
server_key_iv: "bd18b218e643298be15e6e3173e2a2fb"
server_key_version: "encryptionkey@0"
I20260619 12:08:55.503707 21693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4"
format_stamp: "Formatted at 2026-06-19 12:08:55 on dist-test-slave-44vj"
server_key: "494d12ba3504d3b7c7714ff7ac964d82"
server_key_iv: "bd18b218e643298be15e6e3173e2a2fb"
server_key_version: "encryptionkey@0"
I20260619 12:08:55.505174 21693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:55.505992 21708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:55.506193 21693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:55.506301 21693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4"
format_stamp: "Formatted at 2026-06-19 12:08:55 on dist-test-slave-44vj"
server_key: "494d12ba3504d3b7c7714ff7ac964d82"
server_key_iv: "bd18b218e643298be15e6e3173e2a2fb"
server_key_version: "encryptionkey@0"
I20260619 12:08:55.506383 21693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:55.527402 21693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:55.529697 21693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:55.529821 21693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:55.534150 21693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43909
I20260619 12:08:55.534219 21760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43909 every 8 connection(s)
I20260619 12:08:55.534565 21693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:55.535684 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:55.538542 21761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4: Bootstrap starting.
I20260619 12:08:55.539211 21761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:55.539528 21761 log.cc:824] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:55.540421 21761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4: No bootstrap required, opened a new log
I20260619 12:08:55.542218 21761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43909 } }
I20260619 12:08:55.542397 21761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:55.542421 21761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fbf08a6d8a4c39ba2a177dbd71d8b4, State: Initialized, Role: FOLLOWER
I20260619 12:08:55.542557 21761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43909 } }
I20260619 12:08:55.542622 21761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:55.542654 21761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:55.542701 21761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:55.542794 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21693
I20260619 12:08:55.542917 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:55.543115 814 external_mini_cluster.cc:1428] Setting key 636738901f2ef99ded5b65dd86bc67a8
I20260619 12:08:55.543532 21761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43909 } }
I20260619 12:08:55.543949 21761 leader_election.cc:302] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10fbf08a6d8a4c39ba2a177dbd71d8b4; no voters:
I20260619 12:08:55.544166 21761 leader_election.cc:288] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:55.544204 21766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:55.544446 21766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [term 1 LEADER]: Becoming Leader. State: Replica: 10fbf08a6d8a4c39ba2a177dbd71d8b4, State: Running, Role: LEADER
I20260619 12:08:55.544473 21761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:55.544582 21766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43909 } }
I20260619 12:08:55.545054 21767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43909 } } }
I20260619 12:08:55.545145 21767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:55.545346 21768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10fbf08a6d8a4c39ba2a177dbd71d8b4. Latest consensus state: current_term: 1 leader_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbf08a6d8a4c39ba2a177dbd71d8b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43909 } } }
I20260619 12:08:55.545426 21768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:55.546020 21771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:55.546591 21771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:55.548311 21771 catalog_manager.cc:1347] Generated new cluster ID: 6e96b45476e84d4983c8478315c83a50
I20260619 12:08:55.548369 21771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:08:55 dist-test-slave-44vj krb5kdc[21680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:08:55.554317 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:55.543757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:45377 (local address 127.0.0.1:43909)
0619 12:08:55.544003 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:55.544015 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:55.544060 (+ 45us) server_negotiation.cc:407] Connection header received
0619 12:08:55.544235 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:55.544245 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:55.544325 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:55.545411 (+ 1086us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:55.545556 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:55.547005 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:55.547962 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:55.548479 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:55.548581 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:55.551022 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:55.551027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:55.551027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:55.551041 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:55.552727 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:55.553104 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:55.553107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:55.553108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:55.553214 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:55.553422 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:55.553422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:55.553423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:55.553707 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:55.553887 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:55.553978 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 12:08:55.554092 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":123,"threads_started":1}
I20260619 12:08:55.555292 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:55.556279 21771 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:55.557039 21771 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:55.560812 21771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10fbf08a6d8a4c39ba2a177dbd71d8b4: Generated new TSK 0
I20260619 12:08:55.561010 21771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:55.566776 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:55.559775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:48287 (local address 127.0.0.1:43909)
0619 12:08:55.559890 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:55.559892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:55.560029 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:08:55.560093 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:55.560094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:55.560106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:55.560153 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:55.560858 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:55.561507 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:55.562242 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:55.562507 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:55.562562 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:55.563821 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:55.563823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:55.563823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:55.563830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:55.565551 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:55.565977 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:55.565979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:55.565979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:55.566039 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:55.566228 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:55.566229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:55.566230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:55.566352 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:55.566477 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:55.566605 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 12:08:55.566671 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 12:08:55.569552 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21693
W20260619 12:08:55.574529 21678 connection.cc:535] client connection to 127.0.0.1:43909 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43909): Connection reset by peer (error 104)
2026-06-19T12:08:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:55 dist-test-slave-44vj krb5kdc[21792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:55 dist-test-slave-44vj krb5kdc[21792](info): set up 2 sockets
Jun 19 12:08:55 dist-test-slave-44vj krb5kdc[21792](info): commencing operation
krb5kdc: starting...
W20260619 12:08:56.535616 814 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:56 dist-test-slave-44vj krb5kdc[21792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:56.658529 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:56.748521 21805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:56.748801 21805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:56.748839 21805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:56.750223 21805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:56.750278 21805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:56.750308 21805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:56.750325 21805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:56.750367 21805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:56.752221 21805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:56.752521 21805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:56.752760 21805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:56.758085 21813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:56.758085 21811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:56.758147 21810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:56.759393 21805 server_base.cc:1034] running on GCE node
I20260619 12:08:56.759688 21805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:56.760003 21805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:56.761157 21805 hybrid_clock.cc:648] HybridClock initialized: now 1781870936761132 us; error 42 us; skew 500 ppm
Jun 19 12:08:56 dist-test-slave-44vj krb5kdc[21792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870936, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:56.763871 21805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:56.764357 21805 webserver.cc:460] Webserver started at http://127.0.0.1:40059/ using document root <none> and password file <none>
I20260619 12:08:56.764547 21805 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:56.764590 21805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:56.764688 21805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:56.765735 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "a0141043bcec44ff9b628de6c3c4d92a"
format_stamp: "Formatted at 2026-06-19 12:08:56 on dist-test-slave-44vj"
server_key: "074f81a40d1634d75cc9c5bf6bb4271c"
server_key_iv: "b5f97e9a445a9f8348aaf8b8f2b55d9a"
server_key_version: "encryptionkey@0"
I20260619 12:08:56.766039 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "a0141043bcec44ff9b628de6c3c4d92a"
format_stamp: "Formatted at 2026-06-19 12:08:56 on dist-test-slave-44vj"
server_key: "074f81a40d1634d75cc9c5bf6bb4271c"
server_key_iv: "b5f97e9a445a9f8348aaf8b8f2b55d9a"
server_key_version: "encryptionkey@0"
I20260619 12:08:56.767382 21805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:08:56.768313 21820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:56.768555 21805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:56.768656 21805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "a0141043bcec44ff9b628de6c3c4d92a"
format_stamp: "Formatted at 2026-06-19 12:08:56 on dist-test-slave-44vj"
server_key: "074f81a40d1634d75cc9c5bf6bb4271c"
server_key_iv: "b5f97e9a445a9f8348aaf8b8f2b55d9a"
server_key_version: "encryptionkey@0"
I20260619 12:08:56.768767 21805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:56.789659 21805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:56.791949 21805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:56.792088 21805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:56.796785 21805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34921
I20260619 12:08:56.796864 21872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34921 every 8 connection(s)
I20260619 12:08:56.797281 21805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:56.798353 21873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:56.800691 21873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a: Bootstrap starting.
I20260619 12:08:56.801247 21873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:56.801470 21873 log.cc:824] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:56.802346 21873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a: No bootstrap required, opened a new log
I20260619 12:08:56.804000 21873 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0141043bcec44ff9b628de6c3c4d92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 12:08:56.804153 21873 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:56.804175 21873 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0141043bcec44ff9b628de6c3c4d92a, State: Initialized, Role: FOLLOWER
I20260619 12:08:56.804311 21873 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0141043bcec44ff9b628de6c3c4d92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 12:08:56.804391 21873 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:56.804419 21873 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:56.804472 21873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:56.804525 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21805
I20260619 12:08:56.804653 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:56.804852 814 external_mini_cluster.cc:1428] Setting key 2d65ab8e273c1efd76e3ef95419e0d36
I20260619 12:08:56.805167 21873 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0141043bcec44ff9b628de6c3c4d92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 12:08:56.805301 21873 leader_election.cc:302] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0141043bcec44ff9b628de6c3c4d92a; no voters:
I20260619 12:08:56.805503 21873 leader_election.cc:288] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:56.805564 21877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:56.805706 21877 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [term 1 LEADER]: Becoming Leader. State: Replica: a0141043bcec44ff9b628de6c3c4d92a, State: Running, Role: LEADER
I20260619 12:08:56.805836 21873 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:56.805806 21877 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0141043bcec44ff9b628de6c3c4d92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 12:08:56.806559 21880 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0141043bcec44ff9b628de6c3c4d92a. Latest consensus state: current_term: 1 leader_uuid: "a0141043bcec44ff9b628de6c3c4d92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0141043bcec44ff9b628de6c3c4d92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } } }
I20260619 12:08:56.806700 21880 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:56.806955 21878 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0141043bcec44ff9b628de6c3c4d92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0141043bcec44ff9b628de6c3c4d92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } } }
I20260619 12:08:56.807036 21878 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:56.807039 21883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:56.807714 21883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:56.809768 21883 catalog_manager.cc:1347] Generated new cluster ID: b208749d0f964d39a7aa511d9e8b2657
I20260619 12:08:56.809839 21883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:56.812328 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:56.805509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:39423 (local address 127.0.0.1:34921)
0619 12:08:56.805994 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:56.806006 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:56.806039 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:08:56.806168 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:56.806175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:56.806233 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:56.806375 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:56.806633 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:56.808502 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:56.809650 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:56.810330 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:56.810475 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:56.811525 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:56.811530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:56.811531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:56.811793 (+ 262us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:56.811838 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:56.811855 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:56.811949 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:56.812059 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 12:08:56.812091 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":171,"threads_started":1}
I20260619 12:08:56.813452 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:56.818909 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:56.815026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:53611 (local address 127.0.0.1:34921)
0619 12:08:56.815079 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:56.815081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:56.815217 (+ 136us) server_negotiation.cc:407] Connection header received
0619 12:08:56.815290 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:56.815291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:56.815305 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:56.815410 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:56.815525 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:56.816315 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:56.817354 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:56.817737 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:56.817819 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:56.818609 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:56.818611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:56.818612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:56.818617 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:56.818639 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:56.818644 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:56.818709 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:56.818789 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 12:08:56.818806 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:08:56.821831 21883 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:56.822629 21883 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:56.827804 21883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0141043bcec44ff9b628de6c3c4d92a: Generated new TSK 0
I20260619 12:08:56.828048 21883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:56.833743 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21805
W20260619 12:08:56.838187 21790 connection.cc:535] client connection to 127.0.0.1:34921 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34921): Connection reset by peer (error 104)
2026-06-19T12:08:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:56 dist-test-slave-44vj krb5kdc[21904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:56 dist-test-slave-44vj krb5kdc[21904](info): set up 2 sockets
Jun 19 12:08:56 dist-test-slave-44vj krb5kdc[21904](info): commencing operation
krb5kdc: starting...
W20260619 12:08:57.807915 814 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:57 dist-test-slave-44vj krb5kdc[21904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:57.933740 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:58.022364 21917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:58.022647 21917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:58.022672 21917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:58.024336 21917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:58.024469 21917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:58.024502 21917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:58.024523 21917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:58.024544 21917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:58.027073 21917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:58.027490 21917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:58.027818 21917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:58.033746 21922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:58.033756 21923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:58.033975 21925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:58.034852 21917 server_base.cc:1034] running on GCE node
I20260619 12:08:58.035187 21917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:58.035609 21917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:58.036772 21917 hybrid_clock.cc:648] HybridClock initialized: now 1781870938036750 us; error 42 us; skew 500 ppm
Jun 19 12:08:58 dist-test-slave-44vj krb5kdc[21904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870938, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:58.039872 21917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:58.040474 21917 webserver.cc:460] Webserver started at http://127.0.0.1:37387/ using document root <none> and password file <none>
I20260619 12:08:58.040707 21917 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:58.040756 21917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:58.040897 21917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:58.042107 21917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "ef33bfbd1af540e9875587252404107c"
format_stamp: "Formatted at 2026-06-19 12:08:58 on dist-test-slave-44vj"
server_key: "44db7ff21018b1ae2c07243a0653e158"
server_key_iv: "8d02458e34f9a36480203e757f5e6820"
server_key_version: "encryptionkey@0"
I20260619 12:08:58.042491 21917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "ef33bfbd1af540e9875587252404107c"
format_stamp: "Formatted at 2026-06-19 12:08:58 on dist-test-slave-44vj"
server_key: "44db7ff21018b1ae2c07243a0653e158"
server_key_iv: "8d02458e34f9a36480203e757f5e6820"
server_key_version: "encryptionkey@0"
I20260619 12:08:58.044083 21917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:08:58.045113 21932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:58.045428 21917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:08:58.045540 21917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "ef33bfbd1af540e9875587252404107c"
format_stamp: "Formatted at 2026-06-19 12:08:58 on dist-test-slave-44vj"
server_key: "44db7ff21018b1ae2c07243a0653e158"
server_key_iv: "8d02458e34f9a36480203e757f5e6820"
server_key_version: "encryptionkey@0"
I20260619 12:08:58.045621 21917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:58.049443 21917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:58.052156 21917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:58.052306 21917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:58.057190 21917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45645
I20260619 12:08:58.057250 21984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45645 every 8 connection(s)
I20260619 12:08:58.057600 21917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:58.058663 21985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:58.059662 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 21917
I20260619 12:08:58.059793 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:58.059996 814 external_mini_cluster.cc:1428] Setting key 6ef155d83a329b84062d0e102c79cb72
I20260619 12:08:58.061682 21985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c: Bootstrap starting.
I20260619 12:08:58.062423 21985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:58.062726 21985 log.cc:824] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:58.062970 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:58.060634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:41683 (local address 127.0.0.1:45645)
0619 12:08:58.060830 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:58.060840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:58.060877 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:08:58.061552 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:58.061560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:58.061590 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:58.061734 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:58.062467 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:58.062471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:58.062472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:58.062476 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:58.062531 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:58.062546 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:58.062634 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:58.062701 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 12:08:58.062751 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260619 12:08:58.063822 21985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c: No bootstrap required, opened a new log
I20260619 12:08:58.064281 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:58.066125 21985 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef33bfbd1af540e9875587252404107c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260619 12:08:58.066295 21985 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:58.066329 21985 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef33bfbd1af540e9875587252404107c, State: Initialized, Role: FOLLOWER
I20260619 12:08:58.066476 21985 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef33bfbd1af540e9875587252404107c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260619 12:08:58.066569 21985 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:58.066610 21985 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:58.066653 21985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:58.067515 21985 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef33bfbd1af540e9875587252404107c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260619 12:08:58.067606 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:58.065844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:48667 (local address 127.0.0.1:45645)
0619 12:08:58.066493 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:58.066496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:58.066504 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:08:58.066528 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:58.066529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:58.066534 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:58.066589 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:58.067313 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:58.067315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:58.067316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:08:58.067322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:58.067343 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:08:58.067348 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:58.067438 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:58.067497 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:08:58.067517 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622}
I20260619 12:08:58.067625 21985 leader_election.cc:302] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef33bfbd1af540e9875587252404107c; no voters:
I20260619 12:08:58.067906 21985 leader_election.cc:288] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:58.067987 21995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:58.068208 21995 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [term 1 LEADER]: Becoming Leader. State: Replica: ef33bfbd1af540e9875587252404107c, State: Running, Role: LEADER
I20260619 12:08:58.068212 21985 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:58.068351 21995 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef33bfbd1af540e9875587252404107c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260619 12:08:58.068780 21995 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef33bfbd1af540e9875587252404107c. Latest consensus state: current_term: 1 leader_uuid: "ef33bfbd1af540e9875587252404107c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef33bfbd1af540e9875587252404107c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } } }
I20260619 12:08:58.068807 21996 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef33bfbd1af540e9875587252404107c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef33bfbd1af540e9875587252404107c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } } }
I20260619 12:08:58.068879 21995 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:58.068946 21996 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:58.069185 22004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:58.069877 22004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:58.071700 22004 catalog_manager.cc:1347] Generated new cluster ID: c4825cf64147421f9fdedbebc5c33ad6
I20260619 12:08:58.071763 22004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:08:58.078282 22004 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:58.078477 22004 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:58.086654 22004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef33bfbd1af540e9875587252404107c: Generated new TSK 0
I20260619 12:08:58.086839 22004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:58.093147 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 21917
2026-06-19T12:08:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1258 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:58 dist-test-slave-44vj krb5kdc[22016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:58 dist-test-slave-44vj krb5kdc[22016](info): set up 2 sockets
Jun 19 12:08:58 dist-test-slave-44vj krb5kdc[22016](info): commencing operation
krb5kdc: starting...
W20260619 12:08:59.067945 814 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:08:59 dist-test-slave-44vj krb5kdc[22016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:08:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:08:59.194403 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:08:59.282838 22029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:08:59.283308 22029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:08:59.283414 22029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:08:59.284812 22029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:08:59.284875 22029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:08:59.284888 22029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:08:59.284900 22029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:08:59.284911 22029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:08:59.286612 22029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:08:59.286870 22029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:08:59.287110 22029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:08:59.292367 22034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:08:59.292413 22035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:59.292513 22029 server_base.cc:1034] running on GCE node
W20260619 12:08:59.292388 22037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:08:59.292933 22029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:08:59.293260 22029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:08:59.294442 22029 hybrid_clock.cc:648] HybridClock initialized: now 1781870939294410 us; error 53 us; skew 500 ppm
Jun 19 12:08:59 dist-test-slave-44vj krb5kdc[22016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870939, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:08:59.300304 22029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:08:59.300894 22029 webserver.cc:460] Webserver started at http://127.0.0.1:45427/ using document root <none> and password file <none>
I20260619 12:08:59.301151 22029 fs_manager.cc:359] Metadata directory not provided
I20260619 12:08:59.301268 22029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:08:59.301465 22029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:08:59.302656 22029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "4c99aa72f09d4554b062400794f1381f"
format_stamp: "Formatted at 2026-06-19 12:08:59 on dist-test-slave-44vj"
server_key: "f6abeb6f85cf342075fd05368fc3406c"
server_key_iv: "931265ba259eb28e91dacc3594a56a2e"
server_key_version: "encryptionkey@0"
I20260619 12:08:59.303283 22029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "4c99aa72f09d4554b062400794f1381f"
format_stamp: "Formatted at 2026-06-19 12:08:59 on dist-test-slave-44vj"
server_key: "f6abeb6f85cf342075fd05368fc3406c"
server_key_iv: "931265ba259eb28e91dacc3594a56a2e"
server_key_version: "encryptionkey@0"
I20260619 12:08:59.305012 22029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:59.305886 22044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:08:59.306125 22029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:08:59.306227 22029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "4c99aa72f09d4554b062400794f1381f"
format_stamp: "Formatted at 2026-06-19 12:08:59 on dist-test-slave-44vj"
server_key: "f6abeb6f85cf342075fd05368fc3406c"
server_key_iv: "931265ba259eb28e91dacc3594a56a2e"
server_key_version: "encryptionkey@0"
I20260619 12:08:59.306295 22029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:08:59.325037 22029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:08:59.327481 22029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:08:59.327644 22029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:08:59.332089 22029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45125
I20260619 12:08:59.332185 22096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45125 every 8 connection(s)
I20260619 12:08:59.332510 22029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:08:59.333563 22097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:08:59.336306 22097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f: Bootstrap starting.
I20260619 12:08:59.337046 22097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f: Neither blocks nor log segments found. Creating new log.
I20260619 12:08:59.337324 22097 log.cc:824] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:08:59.338267 22097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f: No bootstrap required, opened a new log
I20260619 12:08:59.340142 22097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99aa72f09d4554b062400794f1381f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260619 12:08:59.340317 22097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:08:59.340354 22097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c99aa72f09d4554b062400794f1381f, State: Initialized, Role: FOLLOWER
I20260619 12:08:59.340534 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22029
I20260619 12:08:59.340520 22097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99aa72f09d4554b062400794f1381f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260619 12:08:59.340600 22097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:08:59.340646 22097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:08:59.340659 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:08:59.340696 22097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:08:59.340866 814 external_mini_cluster.cc:1428] Setting key dc81c145afe51e0a5fd72f1ca5e96a46
I20260619 12:08:59.341398 22097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99aa72f09d4554b062400794f1381f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260619 12:08:59.341567 22097 leader_election.cc:302] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c99aa72f09d4554b062400794f1381f; no voters:
I20260619 12:08:59.341774 22097 leader_election.cc:288] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:08:59.341817 22102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:08:59.341958 22102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [term 1 LEADER]: Becoming Leader. State: Replica: 4c99aa72f09d4554b062400794f1381f, State: Running, Role: LEADER
I20260619 12:08:59.342094 22097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:08:59.342124 22102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99aa72f09d4554b062400794f1381f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260619 12:08:59.342759 22103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c99aa72f09d4554b062400794f1381f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99aa72f09d4554b062400794f1381f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } } }
I20260619 12:08:59.342854 22103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [sys.catalog]: This master's current role is: LEADER
I20260619 12:08:59.343372 22104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c99aa72f09d4554b062400794f1381f. Latest consensus state: current_term: 1 leader_uuid: "4c99aa72f09d4554b062400794f1381f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c99aa72f09d4554b062400794f1381f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } } }
I20260619 12:08:59.343521 22104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f [sys.catalog]: This master's current role is: LEADER
W20260619 12:08:59.344061 22118 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:08:59.344144 22118 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:08:59.344193 22111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:08:59.344331 22111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:08:59.346942 22111 catalog_manager.cc:1347] Generated new cluster ID: 600a29db1b5d4272a16dac1369cac98d
I20260619 12:08:59.346992 22111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:08:59 dist-test-slave-44vj krb5kdc[22016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:08:59.353888 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:59.341477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:55389 (local address 127.0.0.1:45125)
0619 12:08:59.341698 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:59.341709 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:59.341753 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:08:59.341901 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:59.341910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:59.341986 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:59.342884 (+ 898us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:59.343028 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:59.344489 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:59.345536 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:59.346022 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:59.346132 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:59.349255 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:59.349260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:59.349261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:59.349277 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:59.352038 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:59.352532 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:59.352535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:59.352536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:59.352682 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:59.352914 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:59.352915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:59.352915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:59.353267 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:59.353453 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:59.353536 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 12:08:59.353659 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260619 12:08:59.355020 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:08:59.364903 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:08:59.356903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:48625 (local address 127.0.0.1:45125)
0619 12:08:59.356944 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:08:59.356946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:08:59.356957 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:08:59.356994 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:08:59.356995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:08:59.357005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:08:59.357085 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:08:59.357705 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:59.358491 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:59.359578 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:08:59.359941 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:08:59.360041 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:08:59.361520 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:08:59.361522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:08:59.361522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:08:59.361532 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:08:59.363290 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:59.363910 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:59.363911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:59.363911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:59.363975 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:08:59.364217 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:08:59.364218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:08:59.364219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:08:59.364365 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:08:59.364455 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:08:59.364729 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 12:08:59.364801 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:08:59.375649 22111 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:08:59.376358 22111 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:08:59.383574 22111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c99aa72f09d4554b062400794f1381f: Generated new TSK 0
I20260619 12:08:59.383899 22111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:08:59.393778 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22029
W20260619 12:08:59.397842 22014 connection.cc:535] client connection to 127.0.0.1:45125 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45125): Connection reset by peer (error 104)
2026-06-19T12:08:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:08:59 dist-test-slave-44vj krb5kdc[22128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:08:59 dist-test-slave-44vj krb5kdc[22128](info): set up 2 sockets
Jun 19 12:08:59 dist-test-slave-44vj krb5kdc[22128](info): commencing operation
krb5kdc: starting...
W20260619 12:09:00.351714 814 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:00 dist-test-slave-44vj krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:09:00.474442 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42345
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:00.566186 22141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:00.566447 22141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:00.566469 22141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:00.567724 22141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:00.567768 22141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:00.567790 22141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:00.567801 22141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:00.567812 22141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:00.569371 22141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42345
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:00.569617 22141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:00.569842 22141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:00.575312 22146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:00.575446 22149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:00.575446 22147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:00.575766 22141 server_base.cc:1034] running on GCE node
I20260619 12:09:00.576030 22141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:00.576373 22141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:00.577574 22141 hybrid_clock.cc:648] HybridClock initialized: now 1781870940577548 us; error 37 us; skew 500 ppm
Jun 19 12:09:00 dist-test-slave-44vj krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870940, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:00.580395 22141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:09:00.580912 22141 webserver.cc:460] Webserver started at http://127.0.0.1:41285/ using document root <none> and password file <none>
I20260619 12:09:00.581158 22141 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:00.581208 22141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:00.581334 22141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:00.582588 22141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "9b04c3144d3d4e57816131cc846c3c9a"
format_stamp: "Formatted at 2026-06-19 12:09:00 on dist-test-slave-44vj"
server_key: "2b61f6a30c301ef8ff3e2cf67dd6a011"
server_key_iv: "ecd11d7530130b7196fedac03a240988"
server_key_version: "encryptionkey@0"
I20260619 12:09:00.583002 22141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "9b04c3144d3d4e57816131cc846c3c9a"
format_stamp: "Formatted at 2026-06-19 12:09:00 on dist-test-slave-44vj"
server_key: "2b61f6a30c301ef8ff3e2cf67dd6a011"
server_key_iv: "ecd11d7530130b7196fedac03a240988"
server_key_version: "encryptionkey@0"
I20260619 12:09:00.584511 22141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:00.585355 22156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:00.585599 22141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:00.585711 22141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "9b04c3144d3d4e57816131cc846c3c9a"
format_stamp: "Formatted at 2026-06-19 12:09:00 on dist-test-slave-44vj"
server_key: "2b61f6a30c301ef8ff3e2cf67dd6a011"
server_key_iv: "ecd11d7530130b7196fedac03a240988"
server_key_version: "encryptionkey@0"
I20260619 12:09:00.585783 22141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:00.593293 22141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:00.595489 22141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:00.595638 22141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:00.599886 22141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42345
I20260619 12:09:00.599943 22208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42345 every 8 connection(s)
I20260619 12:09:00.600317 22141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:00.601482 22209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:00.604367 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a: Bootstrap starting.
I20260619 12:09:00.605052 22209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:00.605357 22209 log.cc:824] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:00.606331 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a: No bootstrap required, opened a new log
I20260619 12:09:00.608156 22209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b04c3144d3d4e57816131cc846c3c9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42345 } }
I20260619 12:09:00.608331 22209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:00.608354 22209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b04c3144d3d4e57816131cc846c3c9a, State: Initialized, Role: FOLLOWER
I20260619 12:09:00.608506 22209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b04c3144d3d4e57816131cc846c3c9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42345 } }
I20260619 12:09:00.608578 22209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:00.608601 22209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:00.608632 22209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:00.609220 22209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b04c3144d3d4e57816131cc846c3c9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42345 } }
I20260619 12:09:00.609323 22209 leader_election.cc:302] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b04c3144d3d4e57816131cc846c3c9a; no voters:
I20260619 12:09:00.609521 22209 leader_election.cc:288] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:00.609647 22212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:00.609792 22209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:00.610107 22212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [term 1 LEADER]: Becoming Leader. State: Replica: 9b04c3144d3d4e57816131cc846c3c9a, State: Running, Role: LEADER
I20260619 12:09:00.610275 22212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b04c3144d3d4e57816131cc846c3c9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42345 } }
I20260619 12:09:00.610491 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22141
I20260619 12:09:00.610642 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:00.610822 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b04c3144d3d4e57816131cc846c3c9a. Latest consensus state: current_term: 1 leader_uuid: "9b04c3144d3d4e57816131cc846c3c9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b04c3144d3d4e57816131cc846c3c9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42345 } } }
I20260619 12:09:00.610888 814 external_mini_cluster.cc:1428] Setting key 014bdc89261a34d2d51406dc57fc8a3b
I20260619 12:09:00.610911 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:00.611013 22213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b04c3144d3d4e57816131cc846c3c9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b04c3144d3d4e57816131cc846c3c9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42345 } } }
I20260619 12:09:00.611086 22213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:00.611230 22221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:00.611852 22221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:00.613934 22221 catalog_manager.cc:1347] Generated new cluster ID: a1996c0234d647c4bad702aba65c4ee4
I20260619 12:09:00.614001 22221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:00.616791 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:00.611608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:44661 (local address 127.0.0.1:42345)
0619 12:09:00.611865 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:00.611877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:00.611916 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:09:00.612083 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:00.612092 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:00.612172 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:00.612337 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:00.612563 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:00.613874 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:00.614684 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:00.615172 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:00.615273 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:00.616045 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:00.616050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:00.616050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:00.616260 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:00.616298 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:00.616311 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:00.616387 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:00.616484 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 12:09:00.616518 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":105,"threads_started":1}
I20260619 12:09:00.617779 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:00.622308 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:00.619270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:39865 (local address 127.0.0.1:42345)
0619 12:09:00.619309 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:00.619311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:00.619448 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:09:00.619532 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:00.619533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:00.619546 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:00.619572 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:00.619765 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:00.620289 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:00.621014 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:00.621288 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:00.621346 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:00.622045 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:00.622047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:00.622047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:00.622052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:00.622070 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:00.622075 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:00.622120 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:00.622188 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 12:09:00.622201 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:00.653589 22221 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:00.654270 22221 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:00.657140 22221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b04c3144d3d4e57816131cc846c3c9a: Generated new TSK 0
I20260619 12:09:00.657356 22221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:00.663113 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22141
W20260619 12:09:00.668000 22126 connection.cc:535] client connection to 127.0.0.1:42345 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42345): Connection reset by peer (error 104)
2026-06-19T12:09:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:00 dist-test-slave-44vj krb5kdc[22240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:00 dist-test-slave-44vj krb5kdc[22240](info): set up 2 sockets
Jun 19 12:09:00 dist-test-slave-44vj krb5kdc[22240](info): commencing operation
krb5kdc: starting...
W20260619 12:09:01.623808 814 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:01 dist-test-slave-44vj krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:09:01.764492 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:01.854802 22253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:01.855077 22253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:01.855114 22253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:01.856781 22253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:01.856870 22253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:01.856894 22253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:01.856909 22253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:01.856925 22253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:01.858798 22253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40091
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:01.859112 22253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:01.859356 22253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:01.865427 22258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:01.865415 22259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:01.865415 22261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:01.865595 22253 server_base.cc:1034] running on GCE node
I20260619 12:09:01.866014 22253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:01.866437 22253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:01.867691 22253 hybrid_clock.cc:648] HybridClock initialized: now 1781870941867671 us; error 36 us; skew 500 ppm
Jun 19 12:09:01 dist-test-slave-44vj krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870941, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:01.870746 22253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:09:01.871309 22253 webserver.cc:460] Webserver started at http://127.0.0.1:37581/ using document root <none> and password file <none>
I20260619 12:09:01.871575 22253 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:01.871629 22253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:01.871775 22253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:01.872905 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "5ad31973b80c475db624311dba5c8e89"
format_stamp: "Formatted at 2026-06-19 12:09:01 on dist-test-slave-44vj"
server_key: "c55a586c53248dbd8ce8c893c33cbff9"
server_key_iv: "57219ac82e041c798153ce1a687c2024"
server_key_version: "encryptionkey@0"
I20260619 12:09:01.873250 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "5ad31973b80c475db624311dba5c8e89"
format_stamp: "Formatted at 2026-06-19 12:09:01 on dist-test-slave-44vj"
server_key: "c55a586c53248dbd8ce8c893c33cbff9"
server_key_iv: "57219ac82e041c798153ce1a687c2024"
server_key_version: "encryptionkey@0"
I20260619 12:09:01.874701 22253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:01.875650 22268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:01.875855 22253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:01.875970 22253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "5ad31973b80c475db624311dba5c8e89"
format_stamp: "Formatted at 2026-06-19 12:09:01 on dist-test-slave-44vj"
server_key: "c55a586c53248dbd8ce8c893c33cbff9"
server_key_iv: "57219ac82e041c798153ce1a687c2024"
server_key_version: "encryptionkey@0"
I20260619 12:09:01.876044 22253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:01.879721 22253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:01.881862 22253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:01.881995 22253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:01.886255 22253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40091
I20260619 12:09:01.886482 22320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40091 every 8 connection(s)
I20260619 12:09:01.886677 22253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:01.887981 22321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:01.890820 22321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89: Bootstrap starting.
I20260619 12:09:01.890867 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22253
I20260619 12:09:01.890982 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:01.891175 814 external_mini_cluster.cc:1428] Setting key ef707246790ea797a6c2e2b9e91695d3
I20260619 12:09:01.891588 22321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:01.892055 22321 log.cc:824] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:01.892781 22321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89: No bootstrap required, opened a new log
I20260619 12:09:01.893914 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:01.891765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:33625 (local address 127.0.0.1:40091)
0619 12:09:01.891979 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:01.891987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:01.892018 (+ 31us) server_negotiation.cc:407] Connection header received
0619 12:09:01.892627 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:01.892635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:01.892665 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:01.892809 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:01.893484 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:01.893488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:01.893488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:01.893491 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:01.893529 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:01.893539 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:01.893612 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:01.893673 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 12:09:01.893710 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260619 12:09:01.895126 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:01.895337 22321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad31973b80c475db624311dba5c8e89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40091 } }
I20260619 12:09:01.895539 22321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:01.895565 22321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad31973b80c475db624311dba5c8e89, State: Initialized, Role: FOLLOWER
I20260619 12:09:01.895699 22321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad31973b80c475db624311dba5c8e89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40091 } }
I20260619 12:09:01.895789 22321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:01.895814 22321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:01.895864 22321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:01.896602 22321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad31973b80c475db624311dba5c8e89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40091 } }
I20260619 12:09:01.896751 22321 leader_election.cc:302] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ad31973b80c475db624311dba5c8e89; no voters:
I20260619 12:09:01.896926 22321 leader_election.cc:288] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:01.897001 22331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:01.897203 22321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:01.897217 22331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [term 1 LEADER]: Becoming Leader. State: Replica: 5ad31973b80c475db624311dba5c8e89, State: Running, Role: LEADER
I20260619 12:09:01.897387 22331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad31973b80c475db624311dba5c8e89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40091 } }
I20260619 12:09:01.897806 22333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ad31973b80c475db624311dba5c8e89. Latest consensus state: current_term: 1 leader_uuid: "5ad31973b80c475db624311dba5c8e89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad31973b80c475db624311dba5c8e89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40091 } } }
I20260619 12:09:01.897830 22332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ad31973b80c475db624311dba5c8e89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad31973b80c475db624311dba5c8e89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40091 } } }
I20260619 12:09:01.897997 22332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:01.898233 22333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:01.898267 22340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:01.899081 22340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:01.899915 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:01.896685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.151:38079 (local address 127.0.0.1:40091)
0619 12:09:01.896742 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:01.896743 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:01.898842 (+ 2099us) server_negotiation.cc:407] Connection header received
0619 12:09:01.898907 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:01.898908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:01.898914 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:01.898962 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:01.899693 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:01.899695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:01.899695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:01.899699 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:01.899717 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:01.899722 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:01.899771 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:01.899828 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:09:01.899843 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:09:01.900946 22340 catalog_manager.cc:1347] Generated new cluster ID: 9437b1bc7b124274a9ae3d0180be93de
I20260619 12:09:01.901018 22340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:01.919986 22340 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:01.920230 22340 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:01.923422 22340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ad31973b80c475db624311dba5c8e89: Generated new TSK 0
I20260619 12:09:01.923619 22340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:01.930063 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22253
2026-06-19T12:09:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1266 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:01 dist-test-slave-44vj krb5kdc[22352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:01 dist-test-slave-44vj krb5kdc[22352](info): set up 2 sockets
Jun 19 12:09:01 dist-test-slave-44vj krb5kdc[22352](info): commencing operation
krb5kdc: starting...
W20260619 12:09:02.891647 814 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:02 dist-test-slave-44vj krb5kdc[22352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:03.024047 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33033
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:03.108353 22365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:03.108577 22365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:03.108600 22365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:03.109810 22365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:03.109866 22365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:03.109880 22365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:03.109892 22365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:03.109905 22365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:03.111498 22365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:33033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:03.111747 22365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:03.111979 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:09:03.116979 22365 server_base.cc:1034] running on GCE node
W20260619 12:09:03.117044 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:03.117056 22371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:03.117234 22370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:03.117501 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:03.117777 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:03.118957 22365 hybrid_clock.cc:648] HybridClock initialized: now 1781870943118935 us; error 38 us; skew 500 ppm
Jun 19 12:09:03 dist-test-slave-44vj krb5kdc[22352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870943, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:03.121825 22365 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:03.122364 22365 webserver.cc:460] Webserver started at http://127.0.203.190:33027/ using document root <none> and password file <none>
I20260619 12:09:03.122601 22365 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:03.122660 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:03.122794 22365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:03.123988 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "092106768dab4b7795d366b59cfb19ac"
format_stamp: "Formatted at 2026-06-19 12:09:03 on dist-test-slave-44vj"
server_key: "53d39280da13cc51ec87e6913a85133a"
server_key_iv: "ab0894e40cadd4bfab50790a5d916630"
server_key_version: "encryptionkey@0"
I20260619 12:09:03.124349 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "092106768dab4b7795d366b59cfb19ac"
format_stamp: "Formatted at 2026-06-19 12:09:03 on dist-test-slave-44vj"
server_key: "53d39280da13cc51ec87e6913a85133a"
server_key_iv: "ab0894e40cadd4bfab50790a5d916630"
server_key_version: "encryptionkey@0"
I20260619 12:09:03.126029 22365 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 12:09:03.127275 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:03.127576 22365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:03.127696 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "092106768dab4b7795d366b59cfb19ac"
format_stamp: "Formatted at 2026-06-19 12:09:03 on dist-test-slave-44vj"
server_key: "53d39280da13cc51ec87e6913a85133a"
server_key_iv: "ab0894e40cadd4bfab50790a5d916630"
server_key_version: "encryptionkey@0"
I20260619 12:09:03.127774 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:03.152184 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:03.154451 22365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:03.154706 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:03.160068 22365 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:33033
I20260619 12:09:03.160140 22432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:33033 every 8 connection(s)
I20260619 12:09:03.160655 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:03.161727 22433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:03.164499 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac: Bootstrap starting.
I20260619 12:09:03.165132 22433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:03.165406 22433 log.cc:824] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:03.166268 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac: No bootstrap required, opened a new log
I20260619 12:09:03.168134 22433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092106768dab4b7795d366b59cfb19ac" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33033 } }
I20260619 12:09:03.168294 22433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:03.168316 22433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 092106768dab4b7795d366b59cfb19ac, State: Initialized, Role: FOLLOWER
I20260619 12:09:03.168452 22433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092106768dab4b7795d366b59cfb19ac" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33033 } }
I20260619 12:09:03.168532 22433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:03.168557 22433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:03.168593 22433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:03.169167 22433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092106768dab4b7795d366b59cfb19ac" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33033 } }
I20260619 12:09:03.169261 22433 leader_election.cc:302] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 092106768dab4b7795d366b59cfb19ac; no voters:
I20260619 12:09:03.169430 22433 leader_election.cc:288] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:03.169593 22436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:03.169687 22433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:03.169924 22436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [term 1 LEADER]: Becoming Leader. State: Replica: 092106768dab4b7795d366b59cfb19ac, State: Running, Role: LEADER
I20260619 12:09:03.170109 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22365
I20260619 12:09:03.170244 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:03.170202 22436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092106768dab4b7795d366b59cfb19ac" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33033 } }
I20260619 12:09:03.170499 814 external_mini_cluster.cc:1428] Setting key 79f9b8aaf039e67bc6adccbb10af3910
I20260619 12:09:03.171504 22438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 092106768dab4b7795d366b59cfb19ac. Latest consensus state: current_term: 1 leader_uuid: "092106768dab4b7795d366b59cfb19ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092106768dab4b7795d366b59cfb19ac" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33033 } } }
I20260619 12:09:03.171782 22438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:03.171948 22437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "092106768dab4b7795d366b59cfb19ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092106768dab4b7795d366b59cfb19ac" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 33033 } } }
I20260619 12:09:03.172022 22437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac [sys.catalog]: This master's current role is: LEADER
W20260619 12:09:03.172276 22453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:03.172326 22453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:03.172896 22454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:03.173218 22454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:03.175164 22454 catalog_manager.cc:1347] Generated new cluster ID: f46aaea2448548f8bfdbbe5910c95a11
I20260619 12:09:03.175218 22454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:03 dist-test-slave-44vj krb5kdc[22352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:03.183598 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:03.171126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58885 (local address 127.0.203.190:33033)
0619 12:09:03.171680 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:03.171692 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:03.171733 (+ 41us) server_negotiation.cc:407] Connection header received
0619 12:09:03.171893 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:03.171902 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:03.171970 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:03.172118 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:03.172912 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:03.174080 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:03.175277 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:03.175988 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:03.176081 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:03.179481 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:03.179489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:03.179489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:03.179506 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:03.181528 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:03.182025 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:03.182029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:03.182030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:03.182169 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:03.182421 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:03.182422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:03.182422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:03.182769 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:03.182917 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:03.183223 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 12:09:03.183334 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":137,"threads_started":1}
I20260619 12:09:03.184790 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:03.192220 22454 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:03.193675 22454 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:03.195091 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:03.186319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33535 (local address 127.0.203.190:33033)
0619 12:09:03.186359 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:03.186361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:03.186627 (+ 266us) server_negotiation.cc:407] Connection header received
0619 12:09:03.186720 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:03.186721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:03.186730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:03.186758 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:03.187553 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:03.188370 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:03.189524 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:03.189883 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:03.189971 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:03.191718 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:03.191721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:03.191721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:03.191731 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:03.193489 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:03.194095 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:03.194096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:03.194097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:03.194177 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:03.194467 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:03.194467 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:03.194468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:03.194628 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:03.194752 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:03.194919 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 12:09:03.194991 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:03.199976 22454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 092106768dab4b7795d366b59cfb19ac: Generated new TSK 0
I20260619 12:09:03.200176 22454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:03.206513 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22365
W20260619 12:09:03.210590 22350 connection.cc:535] client connection to 127.0.203.190:33033 recv error: Network error: failed to read from TLS socket (remote: 127.0.203.190:33033): Connection reset by peer (error 104)
2026-06-19T12:09:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:03 dist-test-slave-44vj krb5kdc[22464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:03 dist-test-slave-44vj krb5kdc[22464](info): set up 2 sockets
Jun 19 12:09:03 dist-test-slave-44vj krb5kdc[22464](info): commencing operation
krb5kdc: starting...
W20260619 12:09:04.171710 814 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:04 dist-test-slave-44vj krb5kdc[22464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:04.303125 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:35999
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:35999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:04.392014 22477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:04.392266 22477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:04.392299 22477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:04.393584 22477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:04.393635 22477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:04.393654 22477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:04.393671 22477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:04.393685 22477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:04.395304 22477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:35999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:35999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:04.395615 22477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:04.395852 22477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:04.401393 22482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:04.401377 22483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:04.401422 22477 server_base.cc:1034] running on GCE node
W20260619 12:09:04.401383 22485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:04.401933 22477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:04.402259 22477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:04.403429 22477 hybrid_clock.cc:648] HybridClock initialized: now 1781870944403390 us; error 69 us; skew 500 ppm
Jun 19 12:09:04 dist-test-slave-44vj krb5kdc[22464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870944, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:04.411713 22477 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:04.412300 22477 webserver.cc:460] Webserver started at http://127.0.203.190:38489/ using document root <none> and password file <none>
I20260619 12:09:04.412546 22477 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:04.412596 22477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:04.412734 22477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:04.414021 22477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "467458f6ed27423493ca344178b239fe"
format_stamp: "Formatted at 2026-06-19 12:09:04 on dist-test-slave-44vj"
server_key: "257f290b1aa3504db4f5c91d5eb52a59"
server_key_iv: "a14e62c72fdcff25e5519a84bb0cecad"
server_key_version: "encryptionkey@0"
I20260619 12:09:04.414387 22477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "467458f6ed27423493ca344178b239fe"
format_stamp: "Formatted at 2026-06-19 12:09:04 on dist-test-slave-44vj"
server_key: "257f290b1aa3504db4f5c91d5eb52a59"
server_key_iv: "a14e62c72fdcff25e5519a84bb0cecad"
server_key_version: "encryptionkey@0"
I20260619 12:09:04.416038 22477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:04.417081 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:04.417348 22477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:04.417459 22477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "467458f6ed27423493ca344178b239fe"
format_stamp: "Formatted at 2026-06-19 12:09:04 on dist-test-slave-44vj"
server_key: "257f290b1aa3504db4f5c91d5eb52a59"
server_key_iv: "a14e62c72fdcff25e5519a84bb0cecad"
server_key_version: "encryptionkey@0"
I20260619 12:09:04.417528 22477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:04.432291 22477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:04.434521 22477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:04.434674 22477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:04.439824 22477 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:35999
I20260619 12:09:04.439916 22544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:35999 every 8 connection(s)
I20260619 12:09:04.440284 22477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:04.441272 22545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:04.443743 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe: Bootstrap starting.
I20260619 12:09:04.444314 22545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:04.444550 22545 log.cc:824] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:04.445370 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe: No bootstrap required, opened a new log
I20260619 12:09:04.446944 22545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467458f6ed27423493ca344178b239fe" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35999 } }
I20260619 12:09:04.447098 22545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:04.447120 22545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467458f6ed27423493ca344178b239fe, State: Initialized, Role: FOLLOWER
I20260619 12:09:04.447242 22545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467458f6ed27423493ca344178b239fe" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35999 } }
I20260619 12:09:04.447324 22545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:04.447355 22545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:04.447453 22545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:04.448015 22545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467458f6ed27423493ca344178b239fe" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35999 } }
I20260619 12:09:04.448107 22545 leader_election.cc:302] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 467458f6ed27423493ca344178b239fe; no voters:
I20260619 12:09:04.448267 22545 leader_election.cc:288] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:04.448381 22548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:04.448508 22545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:04.448616 22548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [term 1 LEADER]: Becoming Leader. State: Replica: 467458f6ed27423493ca344178b239fe, State: Running, Role: LEADER
I20260619 12:09:04.448765 22548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467458f6ed27423493ca344178b239fe" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35999 } }
I20260619 12:09:04.449409 22550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 467458f6ed27423493ca344178b239fe. Latest consensus state: current_term: 1 leader_uuid: "467458f6ed27423493ca344178b239fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467458f6ed27423493ca344178b239fe" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35999 } } }
I20260619 12:09:04.449498 22550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:04.449769 22557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:04.449869 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22477
I20260619 12:09:04.449981 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:04.450204 814 external_mini_cluster.cc:1428] Setting key 0f55032130897a679edfe337749f0073
I20260619 12:09:04.450589 22557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:04.450620 22549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "467458f6ed27423493ca344178b239fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467458f6ed27423493ca344178b239fe" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 35999 } } }
I20260619 12:09:04.450690 22549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:04.452695 22557 catalog_manager.cc:1347] Generated new cluster ID: 39b00e6a7a3a4c35a543614521884bd5
I20260619 12:09:04.452759 22557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:04.457158 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:04.450834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52077 (local address 127.0.203.190:35999)
0619 12:09:04.451203 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:04.451214 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:04.451253 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:09:04.451433 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:04.451440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:04.451506 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:04.451681 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:04.451972 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:04.453337 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:04.454500 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:04.455166 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:04.455685 (+ 519us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:04.456175 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:04.456179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:04.456179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:04.456438 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:04.456483 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:04.456499 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:04.456829 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:04.456877 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 12:09:04.456910 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":81,"threads_started":1}
I20260619 12:09:04.458509 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:04.461086 22557 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:04.461647 22557 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:04.465093 22557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 467458f6ed27423493ca344178b239fe: Generated new TSK 0
I20260619 12:09:04.465282 22557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:04.467648 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:04.460330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47509 (local address 127.0.203.190:35999)
0619 12:09:04.460389 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:04.460391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:04.460609 (+ 218us) server_negotiation.cc:407] Connection header received
0619 12:09:04.460671 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:04.460672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:04.460686 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:04.460739 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:04.460947 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:04.465431 (+ 4484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:04.466192 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:04.466434 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:04.466492 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:04.467348 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:04.467350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:04.467350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:04.467356 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:04.467406 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:04.467411 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:04.467467 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:04.467541 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 12:09:04.467557 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:04.469889 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22477
W20260619 12:09:04.473965 22462 connection.cc:535] client connection to 127.0.203.190:35999 recv error: Network error: failed to read from TLS socket (remote: 127.0.203.190:35999): Connection reset by peer (error 104)
2026-06-19T12:09:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1262 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:04 dist-test-slave-44vj krb5kdc[22576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:04 dist-test-slave-44vj krb5kdc[22576](info): set up 2 sockets
Jun 19 12:09:04 dist-test-slave-44vj krb5kdc[22576](info): commencing operation
krb5kdc: starting...
W20260619 12:09:05.435639 814 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:05 dist-test-slave-44vj krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:05.559098 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:39989
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:40715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:39989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:05.649536 22589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:05.649808 22589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:05.649834 22589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:05.651047 22589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:05.651096 22589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:05.651109 22589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:05.651127 22589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:05.651139 22589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:05.652910 22589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:40715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:39989
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:39989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:05.653169 22589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:05.653388 22589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:05.658586 22594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:05.658586 22595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:05.658802 22597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:05.658898 22589 server_base.cc:1034] running on GCE node
I20260619 12:09:05.659241 22589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:05.659610 22589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:05.660804 22589 hybrid_clock.cc:648] HybridClock initialized: now 1781870945660771 us; error 54 us; skew 500 ppm
Jun 19 12:09:05 dist-test-slave-44vj krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870945, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:05.663628 22589 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:05.664177 22589 webserver.cc:460] Webserver started at http://127.0.203.190:38493/ using document root <none> and password file <none>
I20260619 12:09:05.664409 22589 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:05.664454 22589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:05.664572 22589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:05.665755 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "347ad52979b349d98e634c6338b5c6a8"
format_stamp: "Formatted at 2026-06-19 12:09:05 on dist-test-slave-44vj"
server_key: "96b3cb9260c7bca19ab41e7b39e1a961"
server_key_iv: "74e8ad9e74758a24cd3f156f215e4029"
server_key_version: "encryptionkey@0"
I20260619 12:09:05.666066 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "347ad52979b349d98e634c6338b5c6a8"
format_stamp: "Formatted at 2026-06-19 12:09:05 on dist-test-slave-44vj"
server_key: "96b3cb9260c7bca19ab41e7b39e1a961"
server_key_iv: "74e8ad9e74758a24cd3f156f215e4029"
server_key_version: "encryptionkey@0"
I20260619 12:09:05.667562 22589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:05.668512 22604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:05.668774 22589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:05.668872 22589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "347ad52979b349d98e634c6338b5c6a8"
format_stamp: "Formatted at 2026-06-19 12:09:05 on dist-test-slave-44vj"
server_key: "96b3cb9260c7bca19ab41e7b39e1a961"
server_key_iv: "74e8ad9e74758a24cd3f156f215e4029"
server_key_version: "encryptionkey@0"
I20260619 12:09:05.668939 22589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:05.672601 22589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:05.675187 22589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:05.675333 22589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:05.680086 22589 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:39989
I20260619 12:09:05.680167 22656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:39989 every 8 connection(s)
I20260619 12:09:05.680491 22589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:05.681545 22657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:05.683986 22657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8: Bootstrap starting.
I20260619 12:09:05.684558 22657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:05.684810 22657 log.cc:824] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:05.684994 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22589
I20260619 12:09:05.685127 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:05.685365 814 external_mini_cluster.cc:1428] Setting key bc99e1b84aed968bb09e345113cb834b
I20260619 12:09:05.685642 22657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8: No bootstrap required, opened a new log
I20260619 12:09:05.688042 22657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ad52979b349d98e634c6338b5c6a8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39989 } }
I20260619 12:09:05.688278 22657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:05.688364 22657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347ad52979b349d98e634c6338b5c6a8, State: Initialized, Role: FOLLOWER
I20260619 12:09:05.688505 22657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ad52979b349d98e634c6338b5c6a8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39989 } }
I20260619 12:09:05.688593 22657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:05.688625 22657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:05.688632 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:05.685933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42853 (local address 127.0.203.190:39989)
0619 12:09:05.686143 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:05.686152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:05.686181 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:09:05.686986 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:05.686994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:05.687025 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:05.687167 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:05.687993 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:05.687997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:05.687998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:05.688002 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:05.688044 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:05.688055 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:05.688116 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:05.688185 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 12:09:05.688232 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260619 12:09:05.688675 22657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:05.689483 22657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ad52979b349d98e634c6338b5c6a8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39989 } }
I20260619 12:09:05.689586 22657 leader_election.cc:302] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 347ad52979b349d98e634c6338b5c6a8; no voters:
I20260619 12:09:05.689697 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:05.689786 22657 leader_election.cc:288] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:05.689846 22662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:05.690047 22657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:05.690068 22662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [term 1 LEADER]: Becoming Leader. State: Replica: 347ad52979b349d98e634c6338b5c6a8, State: Running, Role: LEADER
I20260619 12:09:05.690171 22662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ad52979b349d98e634c6338b5c6a8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39989 } }
I20260619 12:09:05.690548 22662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 347ad52979b349d98e634c6338b5c6a8. Latest consensus state: current_term: 1 leader_uuid: "347ad52979b349d98e634c6338b5c6a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ad52979b349d98e634c6338b5c6a8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39989 } } }
I20260619 12:09:05.690623 22663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "347ad52979b349d98e634c6338b5c6a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ad52979b349d98e634c6338b5c6a8" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 39989 } } }
I20260619 12:09:05.690683 22662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:05.690690 22663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:05.691005 22673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:05.691881 22673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:05.693706 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:05.691281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59525 (local address 127.0.203.190:39989)
0619 12:09:05.691320 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:05.691322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:05.692719 (+ 1397us) server_negotiation.cc:407] Connection header received
0619 12:09:05.692789 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:05.692790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:05.692796 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:05.692841 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:05.693475 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:05.693477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:05.693477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:05.693482 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:05.693496 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:05.693501 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:05.693538 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:05.693617 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 12:09:05.693628 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:05.693899 22673 catalog_manager.cc:1347] Generated new cluster ID: 907c07d6cc39409b8150ff8696594f67
I20260619 12:09:05.693972 22673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:05.700577 22673 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:05.700760 22673 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:05.705360 22673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 347ad52979b349d98e634c6338b5c6a8: Generated new TSK 0
I20260619 12:09:05.705574 22673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:05.709960 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22589
2026-06-19T12:09:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1239 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:05 dist-test-slave-44vj krb5kdc[22688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:05 dist-test-slave-44vj krb5kdc[22688](info): set up 2 sockets
Jun 19 12:09:05 dist-test-slave-44vj krb5kdc[22688](info): commencing operation
krb5kdc: starting...
W20260619 12:09:06.667840 814 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:06 dist-test-slave-44vj krb5kdc[22688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:06.802382 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36779
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:36779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:06.887727 22701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:06.887987 22701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:06.888020 22701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:06.889236 22701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:06.889292 22701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:06.889310 22701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:06.889326 22701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:06.889340 22701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:06.890897 22701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:36779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:36779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:06.891176 22701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:06.891428 22701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:06.896473 22709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:06.896466 22707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:06.896420 22706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:06.896814 22701 server_base.cc:1034] running on GCE node
I20260619 12:09:06.897116 22701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:06.897472 22701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:06.898662 22701 hybrid_clock.cc:648] HybridClock initialized: now 1781870946898628 us; error 48 us; skew 500 ppm
Jun 19 12:09:06 dist-test-slave-44vj krb5kdc[22688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870946, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:06.901714 22701 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:06.902276 22701 webserver.cc:460] Webserver started at http://127.0.203.190:35371/ using document root <none> and password file <none>
I20260619 12:09:06.902524 22701 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:06.902577 22701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:06.902747 22701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:06.904032 22701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "d6e8964816f14c02b8b30c07d523ef4c"
format_stamp: "Formatted at 2026-06-19 12:09:06 on dist-test-slave-44vj"
server_key: "badbbc782aff348d0ca75f5d489c1f70"
server_key_iv: "e45c2cb9a23ee5d85f7cef198aacace8"
server_key_version: "encryptionkey@0"
I20260619 12:09:06.904382 22701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "d6e8964816f14c02b8b30c07d523ef4c"
format_stamp: "Formatted at 2026-06-19 12:09:06 on dist-test-slave-44vj"
server_key: "badbbc782aff348d0ca75f5d489c1f70"
server_key_iv: "e45c2cb9a23ee5d85f7cef198aacace8"
server_key_version: "encryptionkey@0"
I20260619 12:09:06.905892 22701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:06.906770 22716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:06.907055 22701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:06.907164 22701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "d6e8964816f14c02b8b30c07d523ef4c"
format_stamp: "Formatted at 2026-06-19 12:09:06 on dist-test-slave-44vj"
server_key: "badbbc782aff348d0ca75f5d489c1f70"
server_key_iv: "e45c2cb9a23ee5d85f7cef198aacace8"
server_key_version: "encryptionkey@0"
I20260619 12:09:06.907239 22701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:06.926890 22701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:06.929770 22701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:06.929929 22701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:06.936986 22701 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:36779
I20260619 12:09:06.937487 22701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:06.937451 22768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:36779 every 8 connection(s)
I20260619 12:09:06.938346 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22701
I20260619 12:09:06.938402 22769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:06.938460 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:06.938652 814 external_mini_cluster.cc:1428] Setting key 90f1965200d51ea7268d757762b6355a
I20260619 12:09:06.942050 22769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c: Bootstrap starting.
I20260619 12:09:06.942835 22769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:06.943143 22769 log.cc:824] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:06.944087 22769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c: No bootstrap required, opened a new log
Jun 19 12:09:06 dist-test-slave-44vj krb5kdc[22688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:06.946062 22769 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e8964816f14c02b8b30c07d523ef4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36779 } }
I20260619 12:09:06.946251 22769 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:06.946297 22769 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e8964816f14c02b8b30c07d523ef4c, State: Initialized, Role: FOLLOWER
I20260619 12:09:06.946434 22769 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e8964816f14c02b8b30c07d523ef4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36779 } }
I20260619 12:09:06.946496 22769 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:06.946528 22769 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:06.946576 22769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:06.947283 22769 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e8964816f14c02b8b30c07d523ef4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36779 } }
I20260619 12:09:06.947446 22769 leader_election.cc:302] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6e8964816f14c02b8b30c07d523ef4c; no voters:
I20260619 12:09:06.947654 22769 leader_election.cc:288] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:06.947697 22774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:06.947844 22774 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [term 1 LEADER]: Becoming Leader. State: Replica: d6e8964816f14c02b8b30c07d523ef4c, State: Running, Role: LEADER
I20260619 12:09:06.947955 22769 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:06.947937 22774 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e8964816f14c02b8b30c07d523ef4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36779 } }
I20260619 12:09:06.948658 22775 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6e8964816f14c02b8b30c07d523ef4c. Latest consensus state: current_term: 1 leader_uuid: "d6e8964816f14c02b8b30c07d523ef4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e8964816f14c02b8b30c07d523ef4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36779 } } }
I20260619 12:09:06.948637 22776 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6e8964816f14c02b8b30c07d523ef4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e8964816f14c02b8b30c07d523ef4c" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 36779 } } }
I20260619 12:09:06.948768 22775 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:06.948776 22776 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:06.949019 22783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:06.949698 22783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:06.951627 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:06.939276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36963 (local address 127.0.203.190:36779)
0619 12:09:06.939476 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:06.939486 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:06.939516 (+ 30us) server_negotiation.cc:407] Connection header received
0619 12:09:06.940110 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:06.940118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:06.940183 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:06.940329 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:06.941161 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:06.942189 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:06.943323 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:06.943885 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:06.944017 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:06.946730 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:06.946737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:06.946737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:06.946752 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:06.949397 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:06.949862 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:06.949867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:06.949867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:06.950011 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:06.950330 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:06.950330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:06.950331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:06.950689 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:06.950817 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:06.951083 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 12:09:06.951411 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260619 12:09:06.951921 22783 catalog_manager.cc:1347] Generated new cluster ID: 9feabc8e72ad4fd0bec7742c53f03bac
I20260619 12:09:06.951968 22783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:06.952778 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:06.962688 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:06.954540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53953 (local address 127.0.203.190:36779)
0619 12:09:06.954582 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:06.954584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:06.954764 (+ 180us) server_negotiation.cc:407] Connection header received
0619 12:09:06.954897 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:06.954898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:06.954910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:06.954937 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:06.955707 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:06.956499 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:06.957551 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:06.957912 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:06.957956 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:06.959762 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:06.959764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:06.959764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:06.959772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:06.961409 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:06.961822 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:06.961823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:06.961824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:06.961892 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:06.962056 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:06.962057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:06.962057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:06.962190 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:06.962275 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:06.962476 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 12:09:06.962568 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:06.969841 22783 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:06.970458 22783 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:06.978525 22783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6e8964816f14c02b8b30c07d523ef4c: Generated new TSK 0
I20260619 12:09:06.978763 22783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:06.988058 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22701
2026-06-19T12:09:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:07 dist-test-slave-44vj krb5kdc[22800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:07 dist-test-slave-44vj krb5kdc[22800](info): set up 2 sockets
Jun 19 12:09:07 dist-test-slave-44vj krb5kdc[22800](info): commencing operation
krb5kdc: starting...
W20260619 12:09:07.963717 814 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:08 dist-test-slave-44vj krb5kdc[22800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:08.096704 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43933
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:43933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:08.184661 22813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:08.184906 22813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:08.184945 22813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:08.186201 22813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:08.186255 22813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:08.186275 22813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:08.186291 22813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:08.186306 22813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:08.187930 22813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:43933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:43933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:08.188202 22813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:08.188441 22813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:08.193868 22818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:08.194044 22821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:08.193852 22819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:08.194237 22813 server_base.cc:1034] running on GCE node
I20260619 12:09:08.194651 22813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:08.195034 22813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:08.196204 22813 hybrid_clock.cc:648] HybridClock initialized: now 1781870948196144 us; error 79 us; skew 500 ppm
Jun 19 12:09:08 dist-test-slave-44vj krb5kdc[22800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870948, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:08.199713 22813 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:08.200311 22813 webserver.cc:460] Webserver started at http://127.0.203.190:44419/ using document root <none> and password file <none>
I20260619 12:09:08.200564 22813 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:08.200623 22813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:08.200809 22813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:08.202191 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "fd2333e11a4c48afbdf257cd75909015"
format_stamp: "Formatted at 2026-06-19 12:09:08 on dist-test-slave-44vj"
server_key: "5306cb2738177608b9511e76875652fb"
server_key_iv: "4b4d6de34bdd5ad31699ede000c9585a"
server_key_version: "encryptionkey@0"
I20260619 12:09:08.202618 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "fd2333e11a4c48afbdf257cd75909015"
format_stamp: "Formatted at 2026-06-19 12:09:08 on dist-test-slave-44vj"
server_key: "5306cb2738177608b9511e76875652fb"
server_key_iv: "4b4d6de34bdd5ad31699ede000c9585a"
server_key_version: "encryptionkey@0"
I20260619 12:09:08.205008 22813 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 12:09:08.206029 22828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:08.206238 22813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:08.206346 22813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "fd2333e11a4c48afbdf257cd75909015"
format_stamp: "Formatted at 2026-06-19 12:09:08 on dist-test-slave-44vj"
server_key: "5306cb2738177608b9511e76875652fb"
server_key_iv: "4b4d6de34bdd5ad31699ede000c9585a"
server_key_version: "encryptionkey@0"
I20260619 12:09:08.206423 22813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:08.231956 22813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:08.234529 22813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:08.234686 22813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:08.239595 22813 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:43933
I20260619 12:09:08.239658 22880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:43933 every 8 connection(s)
I20260619 12:09:08.240052 22813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:08.241082 22881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:08.243089 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22813
I20260619 12:09:08.243220 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:08.243443 814 external_mini_cluster.cc:1428] Setting key 792ce10d123d5c22937b345cad7c78d1
I20260619 12:09:08.243858 22881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015: Bootstrap starting.
I20260619 12:09:08.244607 22881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:08.244892 22881 log.cc:824] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:08.245682 22881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015: No bootstrap required, opened a new log
I20260619 12:09:08.247982 22881 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2333e11a4c48afbdf257cd75909015" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43933 } }
I20260619 12:09:08.248193 22881 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:08.248270 22881 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2333e11a4c48afbdf257cd75909015, State: Initialized, Role: FOLLOWER
I20260619 12:09:08.248423 22881 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2333e11a4c48afbdf257cd75909015" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43933 } }
I20260619 12:09:08.248519 22881 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:08.248561 22881 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:08.248613 22881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:08.249397 22881 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2333e11a4c48afbdf257cd75909015" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43933 } }
I20260619 12:09:08.249526 22881 leader_election.cc:302] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd2333e11a4c48afbdf257cd75909015; no voters:
I20260619 12:09:08.249768 22881 leader_election.cc:288] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:08.249910 22886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:08.250069 22881 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:08.250144 22886 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [term 1 LEADER]: Becoming Leader. State: Replica: fd2333e11a4c48afbdf257cd75909015, State: Running, Role: LEADER
I20260619 12:09:08.250310 22886 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2333e11a4c48afbdf257cd75909015" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43933 } }
I20260619 12:09:08.250784 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd2333e11a4c48afbdf257cd75909015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2333e11a4c48afbdf257cd75909015" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43933 } } }
I20260619 12:09:08.250862 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:08.244132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33341 (local address 127.0.203.190:43933)
0619 12:09:08.244379 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:08.244390 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:08.244423 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:09:08.245071 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:08.245080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:08.245143 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:08.245362 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:08.245728 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:08.246867 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:08.248137 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:08.248807 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:08.248926 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:08.249852 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:08.249858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:08.249858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:08.250102 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:08.250151 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:08.250169 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:08.250495 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:08.250533 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 12:09:08.250627 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":124,"threads_started":1}
I20260619 12:09:08.250972 22887 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd2333e11a4c48afbdf257cd75909015. Latest consensus state: current_term: 1 leader_uuid: "fd2333e11a4c48afbdf257cd75909015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2333e11a4c48afbdf257cd75909015" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 43933 } } }
I20260619 12:09:08.251071 22887 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:08.251830 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015 [sys.catalog]: This master's current role is: LEADER
W20260619 12:09:08.252137 22902 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:08.252193 22902 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:08.252238 22896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:08.252408 22896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:08.252391 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:08.254719 22896 catalog_manager.cc:1347] Generated new cluster ID: 1dca900414aa45288d61c0ac859401ec
I20260619 12:09:08.254786 22896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:08.258044 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:08.254148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51049 (local address 127.0.203.190:43933)
0619 12:09:08.254186 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:08.254188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:08.254346 (+ 158us) server_negotiation.cc:407] Connection header received
0619 12:09:08.254400 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:08.254401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:08.254413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:08.254471 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:08.254608 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:08.255408 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:08.256544 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:08.256888 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:08.256967 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:08.257777 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:08.257778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:08.257778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:08.257784 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:08.257802 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:08.257807 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:08.257856 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:08.257928 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 12:09:08.257956 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:09:08.264842 22896 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:08.265527 22896 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:08.277330 22896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd2333e11a4c48afbdf257cd75909015: Generated new TSK 0
I20260619 12:09:08.277590 22896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:08.282858 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22813
2026-06-19T12:09:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:08 dist-test-slave-44vj krb5kdc[22912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:08 dist-test-slave-44vj krb5kdc[22912](info): set up 2 sockets
Jun 19 12:09:08 dist-test-slave-44vj krb5kdc[22912](info): commencing operation
krb5kdc: starting...
W20260619 12:09:09.287735 814 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:09 dist-test-slave-44vj krb5kdc[22912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:09.419246 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:37529
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:37529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:09.505517 22925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:09.505765 22925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:09.505790 22925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:09.507047 22925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:09.507105 22925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:09.507119 22925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:09.507133 22925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:09.507144 22925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:09.508795 22925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:37529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:37529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:09.509063 22925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:09.509289 22925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:09.514703 22933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:09.514711 22931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:09.514703 22930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:09.515024 22925 server_base.cc:1034] running on GCE node
I20260619 12:09:09.515406 22925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:09.515776 22925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:09.516943 22925 hybrid_clock.cc:648] HybridClock initialized: now 1781870949516925 us; error 40 us; skew 500 ppm
Jun 19 12:09:09 dist-test-slave-44vj krb5kdc[22912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:09.522650 22925 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:09.523263 22925 webserver.cc:460] Webserver started at http://127.0.203.190:33255/ using document root <none> and password file <none>
I20260619 12:09:09.523536 22925 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:09.523590 22925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:09.523738 22925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:09.525012 22925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2"
format_stamp: "Formatted at 2026-06-19 12:09:09 on dist-test-slave-44vj"
server_key: "25c834bff589911007cc274763b2997a"
server_key_iv: "cd39a131265950a4fd55ffcff949197f"
server_key_version: "encryptionkey@0"
I20260619 12:09:09.525390 22925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2"
format_stamp: "Formatted at 2026-06-19 12:09:09 on dist-test-slave-44vj"
server_key: "25c834bff589911007cc274763b2997a"
server_key_iv: "cd39a131265950a4fd55ffcff949197f"
server_key_version: "encryptionkey@0"
I20260619 12:09:09.527024 22925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:09.528178 22940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:09.528473 22925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:09.528585 22925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2"
format_stamp: "Formatted at 2026-06-19 12:09:09 on dist-test-slave-44vj"
server_key: "25c834bff589911007cc274763b2997a"
server_key_iv: "cd39a131265950a4fd55ffcff949197f"
server_key_version: "encryptionkey@0"
I20260619 12:09:09.528678 22925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:09.532684 22925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:09.535637 22925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:09.535852 22925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:09.540776 22925 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:37529
I20260619 12:09:09.540830 22992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:37529 every 8 connection(s)
I20260619 12:09:09.541213 22925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:09.542387 22993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:09.544576 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 22925
I20260619 12:09:09.544703 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:09.544898 814 external_mini_cluster.cc:1428] Setting key 0fe21e95dfa3bb3a2de60d6d4998b350
I20260619 12:09:09.545032 22993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2: Bootstrap starting.
I20260619 12:09:09.545730 22993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:09.546010 22993 log.cc:824] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:09.546821 22993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2: No bootstrap required, opened a new log
I20260619 12:09:09.548228 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:09.545536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52993 (local address 127.0.203.190:37529)
0619 12:09:09.545759 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:09.545769 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:09.545808 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:09:09.546511 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:09.546522 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:09.546560 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:09.546721 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:09.547617 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:09.547621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:09.547622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:09.547670 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:09.547722 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:09.547741 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:09.547835 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:09.547924 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 12:09:09.547986 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260619 12:09:09.549566 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:09.549467 22993 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37529 } }
I20260619 12:09:09.549652 22993 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:09.549695 22993 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef606cd502f54ac0afa37a2ba0dd1cb2, State: Initialized, Role: FOLLOWER
I20260619 12:09:09.549819 22993 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37529 } }
I20260619 12:09:09.549901 22993 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:09.549934 22993 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:09.549984 22993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:09.550645 22993 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37529 } }
I20260619 12:09:09.550755 22993 leader_election.cc:302] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef606cd502f54ac0afa37a2ba0dd1cb2; no voters:
I20260619 12:09:09.550971 22993 leader_election.cc:288] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:09.551008 23002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:09.551126 23002 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [term 1 LEADER]: Becoming Leader. State: Replica: ef606cd502f54ac0afa37a2ba0dd1cb2, State: Running, Role: LEADER
I20260619 12:09:09.551230 23002 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37529 } }
I20260619 12:09:09.551296 22993 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:09.551692 23004 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef606cd502f54ac0afa37a2ba0dd1cb2. Latest consensus state: current_term: 1 leader_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37529 } } }
I20260619 12:09:09.551853 23004 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:09.551712 23003 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef606cd502f54ac0afa37a2ba0dd1cb2" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37529 } } }
I20260619 12:09:09.551978 23003 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:09.552603 23011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:09.553220 23011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:09.553488 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:09.551284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55193 (local address 127.0.203.190:37529)
0619 12:09:09.551330 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:09.551331 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:09.552183 (+ 852us) server_negotiation.cc:407] Connection header received
0619 12:09:09.552257 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:09.552258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:09.552264 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:09.552324 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:09.553162 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:09.553164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:09.553164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:09.553171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:09.553193 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:09.553200 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:09.553269 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:09.553410 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 12:09:09.553427 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:09.555114 23011 catalog_manager.cc:1347] Generated new cluster ID: 5a85d7329903428ca603b3990648cc96
I20260619 12:09:09.555167 23011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:09.565433 23011 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:09.565711 23011 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:09.574534 23011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef606cd502f54ac0afa37a2ba0dd1cb2: Generated new TSK 0
I20260619 12:09:09.574826 23011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:09.577976 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 22925
2026-06-19T12:09:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1295 ms)
[----------] 18 tests from AuthTokenIssuingTest (22939 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:09 dist-test-slave-44vj krb5kdc[23024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:09 dist-test-slave-44vj krb5kdc[23024](info): set up 2 sockets
Jun 19 12:09:09 dist-test-slave-44vj krb5kdc[23024](info): commencing operation
krb5kdc: starting...
W20260619 12:09:10.547693 814 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:10 dist-test-slave-44vj krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:10.672811 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42581
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:42581,127.0.203.189:34895,127.0.203.188:34975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:10.764407 23037 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:09:10.764691 23037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:10.764710 23037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:10.764734 23037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:10.766044 23037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:10.766095 23037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:10.766108 23037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:10.766120 23037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:10.766131 23037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:10.767773 23037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:42581,127.0.203.189:34895,127.0.203.188:34975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:42581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:10.768036 23037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:10.768288 23037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:10.773711 23045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:10.773687 23043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:10.773689 23042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:10.773849 23037 server_base.cc:1034] running on GCE node
I20260619 12:09:10.774425 23037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:10.774799 23037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:10.775960 23037 hybrid_clock.cc:648] HybridClock initialized: now 1781870950775932 us; error 51 us; skew 500 ppm
Jun 19 12:09:10 dist-test-slave-44vj krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:10.778913 23037 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:10.779503 23037 webserver.cc:460] Webserver started at http://127.0.203.190:42153/ using document root <none> and password file <none>
I20260619 12:09:10.779775 23037 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:10.779831 23037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:10.779953 23037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:10.781237 23037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "8e0309bb17a043a88332a038dbdfb389"
format_stamp: "Formatted at 2026-06-19 12:09:10 on dist-test-slave-44vj"
server_key: "6d4494628f2a8a2157674ef623b32582"
server_key_iv: "29435c10e2df73590c8dd6a31767bfb2"
server_key_version: "encryptionkey@0"
I20260619 12:09:10.781657 23037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "8e0309bb17a043a88332a038dbdfb389"
format_stamp: "Formatted at 2026-06-19 12:09:10 on dist-test-slave-44vj"
server_key: "6d4494628f2a8a2157674ef623b32582"
server_key_iv: "29435c10e2df73590c8dd6a31767bfb2"
server_key_version: "encryptionkey@0"
I20260619 12:09:10.783497 23037 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 12:09:10.784611 23052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:10.784890 23037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:10.785001 23037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "8e0309bb17a043a88332a038dbdfb389"
format_stamp: "Formatted at 2026-06-19 12:09:10 on dist-test-slave-44vj"
server_key: "6d4494628f2a8a2157674ef623b32582"
server_key_iv: "29435c10e2df73590c8dd6a31767bfb2"
server_key_version: "encryptionkey@0"
I20260619 12:09:10.785079 23037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:10.802255 23037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:10.804826 23037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:10.804986 23037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:10.810057 23037 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:42581
I20260619 12:09:10.810137 23104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:42581 every 8 connection(s)
I20260619 12:09:10.810504 23037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:10.811542 23105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:10.813201 23105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:10.819082 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 23037
I20260619 12:09:10.819196 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:10.819409 814 external_mini_cluster.cc:1428] Setting key 476ebe48a500a00b7d4d64dc09990fa8
Jun 19 12:09:10 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:10.824103 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.814071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.0.203.190:42581)
0619 12:09:10.814206 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:10.814211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:10.814250 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:09:10.814966 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:10.814968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:10.814986 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:10.815071 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:10.815724 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.816605 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.817492 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.817810 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.817910 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:10.820256 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:10.820264 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:10.820264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:10.820272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:10.822036 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:10.822515 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:10.822522 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:10.822522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:10.822688 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:10.822994 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:10.822996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:10.822996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:10.823311 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:10.823478 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:10.823761 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 12:09:10.823918 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":45,"threads_started":1}
I20260619 12:09:10.824193 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.813882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42581 (local address 127.0.0.1:59400)
0619 12:09:10.814081 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.814133 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:10.814789 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:10.815083 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:10.815086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:10.815451 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:10.815630 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:10.815635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.816714 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.816715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:10.817413 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:10.817414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.817903 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.817904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:10.817968 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:10.818540 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:10.818549 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:10.820173 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:10.822107 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:10.822115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:10.822119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:10.822359 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:10.822783 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:10.822784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:10.822784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:10.822895 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:10.823467 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:10.823474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:10.823567 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 12:09:10.823693 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 12:09:10.824025 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":124,"thread_start_us":92,"threads_started":1}
I20260619 12:09:10.825009 23105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:10.825423 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.825267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42410)
0619 12:09:10.825328 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.825393 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 12:09:10.825604 23055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
I20260619 12:09:10.826048 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.825938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42420)
0619 12:09:10.825991 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.826025 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 12:09:10.826176 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:34895: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.203.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab.
Entry for principal kudu/127.0.203.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab.
I20260619 12:09:10.844395 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:34895
--webserver_interface=127.0.203.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab
--principal=kudu/127.0.203.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:42581,127.0.203.189:34895,127.0.203.188:34975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:09:10.875491 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } attempt: 1
I20260619 12:09:10.876117 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.875978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42434)
0619 12:09:10.876047 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.876086 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:09:10.876653 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.876535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42444)
0619 12:09:10.876595 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.876634 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 12:09:10.876797 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:34895: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
W20260619 12:09:10.933202 23112 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:09:10.933497 23112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:10.933524 23112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:10.933557 23112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:10.934993 23112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:10.935053 23112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:10.935071 23112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:10.935088 23112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:10.935104 23112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:10.936959 23112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:42581,127.0.203.189:34895,127.0.203.188:34975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab
--principal=kudu/127.0.203.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.203.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:10.937258 23112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:10.937498 23112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:10.942782 23118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:10.942773 23120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:10.942773 23117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:10.942862 23112 server_base.cc:1034] running on GCE node
I20260619 12:09:10.943320 23112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:10.943727 23112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:10.944936 23112 hybrid_clock.cc:648] HybridClock initialized: now 1781870950944918 us; error 38 us; skew 500 ppm
I20260619 12:09:10.944941 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } attempt: 2
I20260619 12:09:10.945500 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.945276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42456)
0619 12:09:10.945395 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.945455 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 12:09:10.946015 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.945884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42468)
0619 12:09:10.945966 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.945992 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 12:09:10.946270 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:34895: Network error: Client connection negotiation failed: client connection to 127.0.203.189:34895: connect: Connection refused (error 111)
Jun 19 12:09:10 dist-test-slave-44vj krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:10.948089 23112 init.cc:377] Logged in from keytab as kudu/127.0.203.189@KRBTEST.COM (short username kudu)
I20260619 12:09:10.948598 23112 webserver.cc:460] Webserver started at http://127.0.203.189:37085/ using document root <none> and password file <none>
I20260619 12:09:10.948835 23112 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:10.948894 23112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:10.949034 23112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:10.950181 23112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data/instance:
uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec"
format_stamp: "Formatted at 2026-06-19 12:09:10 on dist-test-slave-44vj"
server_key: "aeb410096bca6ab2d62f82c2c90ebdbd"
server_key_iv: "0459c5f30d410c1a48189c17573b72d1"
server_key_version: "encryptionkey@0"
I20260619 12:09:10.950539 23112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/wal/instance:
uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec"
format_stamp: "Formatted at 2026-06-19 12:09:10 on dist-test-slave-44vj"
server_key: "aeb410096bca6ab2d62f82c2c90ebdbd"
server_key_iv: "0459c5f30d410c1a48189c17573b72d1"
server_key_version: "encryptionkey@0"
I20260619 12:09:10.952101 23112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:10.952972 23127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:10.953218 23112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:10.953329 23112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/wal
uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec"
format_stamp: "Formatted at 2026-06-19 12:09:10 on dist-test-slave-44vj"
server_key: "aeb410096bca6ab2d62f82c2c90ebdbd"
server_key_iv: "0459c5f30d410c1a48189c17573b72d1"
server_key_version: "encryptionkey@0"
I20260619 12:09:10.953400 23112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:10.963670 23112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:10.965858 23112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:10.966002 23112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:10.970404 23112 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.189:34895
I20260619 12:09:10.970479 23179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.189:34895 every 8 connection(s)
I20260619 12:09:10.970839 23112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
I20260619 12:09:10.972029 23180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:10.974138 23180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:10.980175 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 23112
I20260619 12:09:10.980283 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-1/wal/instance
I20260619 12:09:10.980465 814 external_mini_cluster.cc:1428] Setting key 849e3a2341e04098fc05a8e8e3249797
Jun 19 12:09:10 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:10.984969 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.975165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.0.203.190:42581)
0619 12:09:10.975244 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:10.975247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:10.975555 (+ 308us) server_negotiation.cc:407] Connection header received
0619 12:09:10.976156 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:10.976157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:10.976175 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:10.976267 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:10.977088 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.977694 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.978821 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.979066 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.979138 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:10.981544 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:10.981550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:10.981551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:10.981560 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:10.983124 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:10.983804 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:10.983805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:10.983805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:10.983874 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:10.984202 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:10.984202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:10.984203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:10.984338 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:10.984410 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:10.984807 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 12:09:10.984890 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:09:10.985217 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.975140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42581 (local address 127.0.0.1:59410)
0619 12:09:10.975351 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.975434 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:10.976041 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:10.976388 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:10.976392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:10.976708 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:10.976950 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:10.976954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.977828 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.977829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:10.978705 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:10.978708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.979174 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.979175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:10.979293 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:10.979929 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:10.979945 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:10.981432 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:10.983227 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:10.983237 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:10.983239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:10.983603 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:10.983954 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:10.983955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:10.983955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:10.984113 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:10.984415 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:10.984421 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:10.984622 (+ 201us) client_negotiation.cc:769] Sending connection context
0619 12:09:10.984745 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 12:09:10.984942 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260619 12:09:10.985914 23180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.203.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.188@KRBTEST.COM" created.
Jun 19 12:09:10 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:10.994921 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.986312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.0.203.189:34895)
0619 12:09:10.986507 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:10.986513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:10.986534 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:09:10.986552 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:10.986554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:10.986566 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:10.986627 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:10.987029 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.987886 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.989246 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.989676 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.989752 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:10.992217 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:10.992233 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:10.992234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:10.992242 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:10.993784 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:10.994118 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:10.994122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:10.994122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:10.994173 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:10.994322 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:10.994322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:10.994322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:10.994486 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:10.994561 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:10.994686 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 12:09:10.994791 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":164,"thread_start_us":118,"threads_started":1}
I20260619 12:09:10.994966 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.986232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42478)
0619 12:09:10.986308 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.986320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:10.986366 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:10.986674 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:10.986676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:10.986838 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:10.986929 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:10.986932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.987994 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.987996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:10.989132 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:10.989137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:10.989800 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:10.989801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:10.989886 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:10.990349 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:10.990354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:10.992109 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:10.993874 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:10.993875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:10.993876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:10.994050 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:10.994223 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:10.994224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:10.994225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:10.994275 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:10.994548 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:10.994549 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:10.994590 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 12:09:10.994653 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 12:09:10.994849 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":89}
I20260619 12:09:10.996064 23180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:10.996507 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.996371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48010)
0619 12:09:10.996420 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.996480 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 12:09:10.996675 23131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
I20260619 12:09:10.997249 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:10.997143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48012)
0619 12:09:10.997197 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:10.997223 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:09:10.997354 23180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:34975: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab.
Entry for principal kudu/127.0.203.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab.
I20260619 12:09:11.006606 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:34975
--webserver_interface=127.0.203.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab
--principal=kudu/127.0.203.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:42581,127.0.203.189:34895,127.0.203.188:34975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:09:11.037494 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } attempt: 3
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:11.046590 23180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } attempt: 1
I20260619 12:09:11.047142 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.046937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48028)
0619 12:09:11.047019 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.047080 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:09:11.047482 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.038147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42490)
0619 12:09:11.038250 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.038272 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:11.038359 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:11.038596 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:11.038598 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:11.039001 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:11.039092 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.039096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.040221 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.040223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.040920 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.040924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.041544 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.041546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.041639 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.042252 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:11.042260 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:11.044052 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:11.046100 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.046101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.046102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.046341 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.046660 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.046661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.046661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.046744 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.047114 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:11.047116 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:11.047164 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 12:09:11.047229 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:09:11.047355 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 12:09:11.047780 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.047519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48038)
0619 12:09:11.047714 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.047744 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":134,"threads_started":1}
W20260619 12:09:11.047899 23180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:34975: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
I20260619 12:09:11.047914 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.038250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.0.203.189:34895)
0619 12:09:11.038353 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.038356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.038374 (+ 18us) server_negotiation.cc:407] Connection header received
0619 12:09:11.038431 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.038433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.038451 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.038514 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.039197 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.040075 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.041035 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.041413 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.041497 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.044180 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.044182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.044182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.044194 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.045957 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.046438 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.046439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.046439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.046529 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.046842 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.046844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.046845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.047025 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.047092 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.047485 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.047826 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:09:11.048636 23105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:11.049145 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.048963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48048)
0619 12:09:11.049074 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.049116 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260619 12:09:11.049782 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.049579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48064)
0619 12:09:11.049623 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.049739 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 12:09:11.049976 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:34975: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
I20260619 12:09:11.081215 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } attempt: 1
I20260619 12:09:11.082127 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.081789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48078)
0619 12:09:11.081935 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.082003 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 12:09:11.082777 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.082649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48080)
0619 12:09:11.082717 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.082748 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 12:09:11.082926 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:34975: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
W20260619 12:09:11.101610 23187 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:09:11.101909 23187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:11.101927 23187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:11.101949 23187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:11.103255 23187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:11.103309 23187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:11.103322 23187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:11.103333 23187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:11.103344 23187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:11.105113 23187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:42581,127.0.203.189:34895,127.0.203.188:34975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab
--principal=kudu/127.0.203.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:34975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.203.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:11.105401 23187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:11.105638 23187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:11.111404 23196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:11.111418 23194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:11.111404 23193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:11.111835 23187 server_base.cc:1034] running on GCE node
I20260619 12:09:11.112131 23187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:11.112514 23187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:11.113688 23187 hybrid_clock.cc:648] HybridClock initialized: now 1781870951113652 us; error 58 us; skew 500 ppm
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:11.116101 23180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } attempt: 2
I20260619 12:09:11.116607 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.116452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48094)
0619 12:09:11.116531 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.116568 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:09:11.116647 23187 init.cc:377] Logged in from keytab as kudu/127.0.203.188@KRBTEST.COM (short username kudu)
I20260619 12:09:11.117060 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.116948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48108)
0619 12:09:11.116999 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.117035 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 12:09:11.117166 23180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:34975: Network error: Client connection negotiation failed: client connection to 127.0.203.188:34975: connect: Connection refused (error 111)
I20260619 12:09:11.117338 23187 webserver.cc:460] Webserver started at http://127.0.203.188:36195/ using document root <none> and password file <none>
I20260619 12:09:11.117579 23187 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:11.117630 23187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:11.117779 23187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:11.118978 23187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data/instance:
uuid: "3c01054c0fdb4936a70990970fae7129"
format_stamp: "Formatted at 2026-06-19 12:09:11 on dist-test-slave-44vj"
server_key: "a365ec0f1aacdd825158b4c68ddb78f1"
server_key_iv: "0793287b3a7229a2f00b9a2d6a6524a8"
server_key_version: "encryptionkey@0"
I20260619 12:09:11.119338 23187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/wal/instance:
uuid: "3c01054c0fdb4936a70990970fae7129"
format_stamp: "Formatted at 2026-06-19 12:09:11 on dist-test-slave-44vj"
server_key: "a365ec0f1aacdd825158b4c68ddb78f1"
server_key_iv: "0793287b3a7229a2f00b9a2d6a6524a8"
server_key_version: "encryptionkey@0"
I20260619 12:09:11.120882 23187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:11.121865 23203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:11.122145 23187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:11.122259 23187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/wal
uuid: "3c01054c0fdb4936a70990970fae7129"
format_stamp: "Formatted at 2026-06-19 12:09:11 on dist-test-slave-44vj"
server_key: "a365ec0f1aacdd825158b4c68ddb78f1"
server_key_iv: "0793287b3a7229a2f00b9a2d6a6524a8"
server_key_version: "encryptionkey@0"
I20260619 12:09:11.122339 23187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:11.135123 23187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:11.137796 23187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:11.137974 23187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:11.143065 23187 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.188:34975
I20260619 12:09:11.143143 23255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.188:34975 every 8 connection(s)
I20260619 12:09:11.143621 23187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
I20260619 12:09:11.144778 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:11.146553 23256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:11.152884 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 23187
I20260619 12:09:11.153004 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870673623844-814-0/minicluster-data/master-2/wal/instance
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:11.153184 814 external_mini_cluster.cc:1428] Setting key 894fc6253086f7a87b729eeca7f152db
I20260619 12:09:11.157702 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.147508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:42581 (local address 127.0.0.1:59412)
0619 12:09:11.147760 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.147809 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:11.148336 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:11.148587 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:11.148591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:11.148871 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:11.149084 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.149087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.149873 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.149875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.150742 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.150745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.151181 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.151182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.151312 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.151928 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:11.151938 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:11.153765 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:11.155756 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.155770 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.155772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.156165 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.156472 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.156473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.156473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.156644 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.156932 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:11.156937 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:11.157103 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 12:09:11.157201 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:09:11.157429 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":132,"threads_started":1}
I20260619 12:09:11.157788 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.147520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.0.203.190:42581)
0619 12:09:11.147580 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.147582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.147929 (+ 347us) server_negotiation.cc:407] Connection header received
0619 12:09:11.148444 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.148446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.148459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.148524 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.149202 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.149746 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.150850 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.151061 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.151131 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.153899 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.153901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.153902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.153912 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.155648 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.156308 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.156310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.156311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.156387 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.156709 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.156710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.156710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.156855 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.156936 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.157590 (+ 654us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.157691 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:09:11.158064 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } attempt: 2
I20260619 12:09:11.158684 23256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:11.163602 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.153636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.0.203.188:34975)
0619 12:09:11.153788 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.153793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.153850 (+ 57us) server_negotiation.cc:407] Connection header received
0619 12:09:11.153925 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.153929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.153945 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.154043 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.154664 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.155741 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.156769 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.157209 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.157292 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.160255 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.160264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.160264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.160271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.162063 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.162556 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.162560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.162560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.162639 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.162813 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.162814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.162814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.162994 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.163152 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.163400 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.163487 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260619 12:09:11.166900 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.159011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:34895 (local address 127.0.0.1:42500)
0619 12:09:11.159062 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.159071 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:11.159106 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:11.159250 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:11.159251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:11.159424 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:11.159498 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.159500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.160348 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.160350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.161101 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.161103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.162022 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.162023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.162113 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.162516 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:11.162520 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:11.163854 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:11.165688 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.165689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.165690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.166016 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.166226 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.166227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.166227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.166289 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.166529 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:11.166530 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:11.166588 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 12:09:11.166661 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:09:11.166784 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 12:09:11.166970 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.159076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.0.203.189:34895)
0619 12:09:11.159117 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.159119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.159127 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:11.159157 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.159158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.159167 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.159229 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.159578 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.160258 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.161193 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.161422 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.161490 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.163966 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.163969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.163969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.163977 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.165589 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.166098 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.166099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.166100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.166161 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.166341 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.166342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.166342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.166469 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.166530 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.166751 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.166867 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:11.167395 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.158324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48116)
0619 12:09:11.158375 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.158388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:11.158429 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:11.158649 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:11.158650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:11.158801 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:11.158849 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.158851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.159709 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.159710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.160477 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.160479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.160961 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.160963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.161042 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.161502 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:11.161506 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:11.162859 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:11.166054 (+ 3195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.166056 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.166057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.166279 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.166708 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.166709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.166710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.166780 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.167056 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:11.167057 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:11.167110 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 12:09:11.167153 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 12:09:11.167270 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 12:09:11.167785 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.158364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48116 (local address 127.0.203.188:34975)
0619 12:09:11.158526 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.158529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.158538 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:11.158556 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.158557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.158569 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.158640 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.158929 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.159620 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.160561 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.160873 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.160936 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.162952 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.162953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.162954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.162961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.165970 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.166561 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.166562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.166562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.166628 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.166850 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.166851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.166851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.166974 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.167535 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.167578 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.167667 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260619 12:09:11.168503 23256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:11.170110 23105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389: Bootstrap starting.
I20260619 12:09:11.170812 23105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:11.171296 23105 log.cc:824] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:11.172506 23105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389: No bootstrap required, opened a new log
I20260619 12:09:11.173388 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.164570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.0.203.190:42581)
0619 12:09:11.164628 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.164631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.164699 (+ 68us) server_negotiation.cc:407] Connection header received
0619 12:09:11.164779 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.164780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.164792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.164847 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.165461 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.166180 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.167068 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.167311 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.167346 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.170249 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.170251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.170251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.170260 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.171930 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.172386 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.172388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.172388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.172491 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.172802 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.172803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.172803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.172968 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.173144 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.173187 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.173298 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:09:11.174595 23105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.174728 23105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0309bb17a043a88332a038dbdfb389, State: Initialized, Role: FOLLOWER
I20260619 12:09:11.174790 23105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.175185 23105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:11.175140 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.175268 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:09:11.176977 23274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:11.177093 23274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:11.177783 23037 master_runner.cc:186] Error getting master registration for 127.0.203.189:34895: OK, instance_id { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" instance_seqno: 1781870950969120 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:09:11.180140 23037 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:09:11.180279 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.168911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48122)
0619 12:09:11.168992 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.169002 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:11.169046 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:11.169165 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:11.169166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:11.169304 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:11.169341 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.169343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.170193 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.170194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.171446 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.171450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.175470 (+ 4020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.175471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.175548 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.175973 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:11.175978 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:11.177588 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:11.179199 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.179200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.179200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.179433 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.179605 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.179605 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.179606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.179651 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.179978 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:11.179979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:11.180017 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 12:09:11.180067 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:09:11.180164 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 12:09:11.180649 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.168960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.0.203.188:34975)
0619 12:09:11.168995 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.168997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.169031 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:09:11.169096 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.169097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.169107 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.169156 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.169400 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.169973 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.171533 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.171879 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.171950 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.177677 (+ 5727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.177678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.177679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.177686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.179122 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.179505 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.179505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.179506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.179551 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.179700 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.179701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.179701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.179896 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.179957 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.180487 (+ 530us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.180569 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 12:09:11.182375 23256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: Bootstrap starting.
I20260619 12:09:11.182588 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.174158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.0.203.189:34895)
0619 12:09:11.174210 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.174213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.174222 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:11.174249 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.174250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.174262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.174340 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.174854 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.175546 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.176759 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.177058 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.177120 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.180195 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.180197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.180197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.180204 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.181627 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.181913 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.181914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.181915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.181970 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.182121 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.182122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.182123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.182235 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.182392 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.182426 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.182498 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:09:11.183132 23256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:11.183544 23256 log.cc:824] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:11.184624 23256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: No bootstrap required, opened a new log
I20260619 12:09:11.185012 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:11.186285 23079 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8e0309bb17a043a88332a038dbdfb389"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59426
I20260619 12:09:11.186486 23079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 12:09:11.186546 23079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:11.186683 23256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.186861 23256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c01054c0fdb4936a70990970fae7129, State: Initialized, Role: FOLLOWER
I20260619 12:09:11.186959 23256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.187428 23256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:11.187347 23277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.187659 23277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:11.187805 23079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.188146 23079 leader_election.cc:288] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [CANDIDATE]: Term 1 election: Requested vote from peers 8d7715fd27cd4e38ba21bd7d8bc71bec (127.0.203.189:34895), 3c01054c0fdb4936a70990970fae7129 (127.0.203.188:34975)
I20260619 12:09:11.188956 23230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e0309bb17a043a88332a038dbdfb389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3c01054c0fdb4936a70990970fae7129"
I20260619 12:09:11.189102 23230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:11.190169 23230 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e0309bb17a043a88332a038dbdfb389 in term 1.
I20260619 12:09:11.190505 23056 leader_election.cc:302] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c01054c0fdb4936a70990970fae7129, 8e0309bb17a043a88332a038dbdfb389; no voters:
I20260619 12:09:11.191141 23263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:11.191411 23263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [term 1 LEADER]: Becoming Leader. State: Replica: 8e0309bb17a043a88332a038dbdfb389, State: Running, Role: LEADER
I20260619 12:09:11.191534 23263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.192049 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e0309bb17a043a88332a038dbdfb389. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.192131 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: This master's current role is: LEADER
W20260619 12:09:11.195945 23295 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:11.196103 23295 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:11.193985 23187 master_runner.cc:186] Error getting master registration for 127.0.203.189:34895: OK, instance_id { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" instance_seqno: 1781870950969120 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:09:11.196784 23187 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:09:11.200022 23298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:11.202499 23298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:11.204487 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.190334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48132 (local address 127.0.203.188:34975)
0619 12:09:11.196897 (+ 6563us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.196899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.196933 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:09:11.196956 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.196957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.196996 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.197074 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.198128 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.198866 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.199879 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.200144 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.200215 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.201464 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.201466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.201467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.201474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.203145 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.203536 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.203538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.203538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.203613 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.203779 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.203780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.203781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.203933 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.204045 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.204302 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.204382 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6534}
I20260619 12:09:11.204770 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.190024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59428 (local address 127.0.203.190:42581)
0619 12:09:11.190084 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.190086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.190342 (+ 256us) server_negotiation.cc:407] Connection header received
0619 12:09:11.190443 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.190444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.190456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.190485 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.191460 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.192149 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.193129 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.193408 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.193448 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.196330 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.196332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.196332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.196340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.198759 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.203616 (+ 4857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.203619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.203620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.203744 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.204062 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.204063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.204063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.204225 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.204319 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.204583 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.204690 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:09:11.206187 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.190436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.0.203.189:34895)
0619 12:09:11.190704 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.190706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.190833 (+ 127us) server_negotiation.cc:407] Connection header received
0619 12:09:11.190935 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.190935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.190947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.190978 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.192398 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.193182 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.194291 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.194598 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.194641 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.196294 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.196295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.196296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.196304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.198135 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.205046 (+ 6911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.205049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.205049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.205134 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.205387 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.205389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.205390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.205598 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.205706 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.205980 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.206080 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238}
I20260619 12:09:11.208266 23230 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [term 1 FOLLOWER]: Refusing update from remote peer 8e0309bb17a043a88332a038dbdfb389: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:11.208310 23180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } attempt: 3
I20260619 12:09:11.208623 23263 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:11.210161 23277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e0309bb17a043a88332a038dbdfb389. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.210247 23277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 12:09:11 dist-test-slave-44vj krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
I20260619 12:09:11.218911 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.208590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:34975 (local address 127.0.0.1:48144)
0619 12:09:11.208688 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:11.208699 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:11.208737 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:11.209244 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:11.209245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:11.209428 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:11.209491 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.209493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.210563 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.210565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.211463 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:11.211466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.211993 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.211994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:11.212073 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.212575 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:11.212580 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:11.214951 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:11.217171 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.217173 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.217173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.217383 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.217589 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:11.217602 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:11.217602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:11.217658 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:11.218578 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:11.218579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:11.218641 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 12:09:11.218686 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:09:11.218802 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 12:09:11.219408 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:11.208809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.0.203.188:34975)
0619 12:09:11.209090 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.209092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.209099 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:11.209134 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.209135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.209145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.209199 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.209586 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.210397 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.211581 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:11.211912 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:11.211984 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:11.215047 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:11.215048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:11.215049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:11.215057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:11.217070 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.217464 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.217465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.217465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.217523 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:11.217722 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:11.217723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:11.217723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:11.218500 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:11.219154 (+ 654us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:11.219199 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 12:09:11.219282 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260619 12:09:11.221298 23180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: Bootstrap starting.
I20260619 12:09:11.222062 23180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:11.222345 23180 log.cc:824] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:11.222447 23300 mvcc.cc:205] Tried to move back new op lower bound from 7298543416145412096 to 7298543416081506304. Current Snapshot: MvccSnapshot[applied={T|T < 7298543416145412096}]
I20260619 12:09:11.223160 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.223269 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:11.223489 23277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.223591 23277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:11.223659 23301 mvcc.cc:205] Tried to move back new op lower bound from 7298543416145412096 to 7298543416081506304. Current Snapshot: MvccSnapshot[applied={T|T < 7298543416145412096}]
I20260619 12:09:11.231978 23298 catalog_manager.cc:1347] Generated new cluster ID: 03a295340bc247c99a504a65fa14a5d7
I20260619 12:09:11.232214 23298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:11.232829 23180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: No bootstrap required, opened a new log
I20260619 12:09:11.235603 23180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.235810 23180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7715fd27cd4e38ba21bd7d8bc71bec, State: Initialized, Role: FOLLOWER
I20260619 12:09:11.235906 23180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } }
I20260619 12:09:11.236399 23180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:11.236573 23154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e0309bb17a043a88332a038dbdfb389" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec"
I20260619 12:09:11.237175 23154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:11.238296 23154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e0309bb17a043a88332a038dbdfb389 in term 1.
I20260619 12:09:11.238623 23153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [term 1 FOLLOWER]: Refusing update from remote peer 8e0309bb17a043a88332a038dbdfb389: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:11.243394 23296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:11.246089 23307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.246264 23307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:11.246316 23307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e0309bb17a043a88332a038dbdfb389. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.246369 23307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:11.246397 23307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.246440 23307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:09:11.247207 23318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:11.251420 23318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:11.250206 23320 mvcc.cc:205] Tried to move back new op lower bound from 7298543416145412096 to 7298543416081506304. Current Snapshot: MvccSnapshot[applied={T|T < 7298543416145412096}]
I20260619 12:09:11.255875 23298 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:11.256641 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e0309bb17a043a88332a038dbdfb389" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0309bb17a043a88332a038dbdfb389" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 42581 } } peers { permanent_uuid: "8d7715fd27cd4e38ba21bd7d8bc71bec" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 34895 } } peers { permanent_uuid: "3c01054c0fdb4936a70990970fae7129" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 34975 } } }
I20260619 12:09:11.256762 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:11.256826 23298 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:11.294508 23298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e0309bb17a043a88332a038dbdfb389: Generated new TSK 0
I20260619 12:09:11.294790 23298 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:09:11.327452 23108 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.324734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.0.203.190:42581)
0619 12:09:11.324786 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.324788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.325778 (+ 990us) server_negotiation.cc:407] Connection header received
0619 12:09:11.326128 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.326130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.326143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.326173 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.327311 (+ 1138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59436: BlockingRecv error: recv got EOF from 127.0.0.1:59436 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 12:09:11.327888 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.325040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.0.203.189:34895)
0619 12:09:11.325082 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.325084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.325883 (+ 799us) server_negotiation.cc:407] Connection header received
0619 12:09:11.326419 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.326421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.326434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.326464 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.327830 (+ 1366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42546: BlockingRecv error: recv got EOF from 127.0.0.1:42546 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 12:09:11.328308 23261 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.325264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48152 (local address 127.0.203.188:34975)
0619 12:09:11.325308 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.325310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.326538 (+ 1228us) server_negotiation.cc:407] Connection header received
0619 12:09:11.326765 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.326766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.326776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.326802 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.328252 (+ 1450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48152: BlockingRecv error: recv got EOF from 127.0.0.1:48152 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 12:09:11.330862 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.330115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.0.203.189:34895)
0619 12:09:11.330153 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.330155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.330363 (+ 208us) server_negotiation.cc:407] Connection header received
0619 12:09:11.330468 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.330469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.330488 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.330516 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.330803 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42558: BlockingRecv error: recv got EOF from 127.0.0.1:42558 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 12:09:11.333981 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.333156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.0.203.189:34895)
0619 12:09:11.333192 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.333193 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.333404 (+ 211us) server_negotiation.cc:407] Connection header received
0619 12:09:11.333495 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.333496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.333507 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.333533 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.333917 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42572: BlockingRecv error: recv got EOF from 127.0.0.1:42572 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 12:09:11.338124 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.337247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42586 (local address 127.0.203.189:34895)
0619 12:09:11.337291 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.337293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.337545 (+ 252us) server_negotiation.cc:407] Connection header received
0619 12:09:11.337640 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.337641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.337653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.337680 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.338070 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42586: BlockingRecv error: recv got EOF from 127.0.0.1:42586 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 12:09:11.344501 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.343576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.0.203.189:34895)
0619 12:09:11.343623 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.343624 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.343915 (+ 291us) server_negotiation.cc:407] Connection header received
0619 12:09:11.344015 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.344017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.344028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.344057 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.344439 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42588: BlockingRecv error: recv got EOF from 127.0.0.1:42588 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 12:09:11.354947 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.353945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.0.203.189:34895)
0619 12:09:11.353992 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.353995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.354299 (+ 304us) server_negotiation.cc:407] Connection header received
0619 12:09:11.354407 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.354409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.354422 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.354452 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.354880 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42592: BlockingRecv error: recv got EOF from 127.0.0.1:42592 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 12:09:11.375347 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.374424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.0.203.189:34895)
0619 12:09:11.374470 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.374472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.374789 (+ 317us) server_negotiation.cc:407] Connection header received
0619 12:09:11.374881 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.374883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.374895 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.374921 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.375285 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42606: BlockingRecv error: recv got EOF from 127.0.0.1:42606 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 12:09:11.409680 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.408920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.0.203.189:34895)
0619 12:09:11.408983 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.408985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.409154 (+ 169us) server_negotiation.cc:407] Connection header received
0619 12:09:11.409227 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.409228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.409239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.409298 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.409622 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42614: BlockingRecv error: recv got EOF from 127.0.0.1:42614 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 12:09:11.476151 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.475266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42618 (local address 127.0.203.189:34895)
0619 12:09:11.475418 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.475421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.475532 (+ 111us) server_negotiation.cc:407] Connection header received
0619 12:09:11.475612 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.475613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.475626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.475675 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.476089 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42618: BlockingRecv error: recv got EOF from 127.0.0.1:42618 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260619 12:09:11.606423 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.605549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.0.203.189:34895)
0619 12:09:11.605645 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.605647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.605798 (+ 151us) server_negotiation.cc:407] Connection header received
0619 12:09:11.605834 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.605836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.605852 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.605926 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.606360 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42624: BlockingRecv error: recv got EOF from 127.0.0.1:42624 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 12:09:11.864637 23184 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:11.863966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.0.203.189:34895)
0619 12:09:11.864024 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:11.864026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:11.864036 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:09:11.864066 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:11.864066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:11.864077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:11.864157 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:11.864585 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42632: BlockingRecv error: recv got EOF from 127.0.0.1:42632 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:09:12.196653 23295 catalog_manager.cc:1259] Loaded cluster ID: 03a295340bc247c99a504a65fa14a5d7
I20260619 12:09:12.196738 23295 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: loading cluster ID for follower catalog manager: success
I20260619 12:09:12.197580 23295 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: acquiring CA information for follower catalog manager: success
I20260619 12:09:12.197858 23295 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c01054c0fdb4936a70990970fae7129: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:09:12.251981 23318 catalog_manager.cc:1259] Loaded cluster ID: 03a295340bc247c99a504a65fa14a5d7
I20260619 12:09:12.252060 23318 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: loading cluster ID for follower catalog manager: success
I20260619 12:09:12.252823 23318 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: acquiring CA information for follower catalog manager: success
I20260619 12:09:12.253072 23318 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d7715fd27cd4e38ba21bd7d8bc71bec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:09:12.381371 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:12.378078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.0.203.189:34895)
0619 12:09:12.378229 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:12.378231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:12.378247 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:09:12.378286 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:12.378287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:12.378357 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:12.378491 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:12.378684 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.379282 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.380262 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.380489 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.380567 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:12.380818 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:12.380984 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:12.381070 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:12.381205 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 12:09:12.381232 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260619 12:09:12.386683 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:12.383655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48160 (local address 127.0.203.188:34975)
0619 12:09:12.383816 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:12.383819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:12.383829 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:09:12.383914 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:12.383915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:12.383962 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:12.384040 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:12.384228 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.384788 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.385659 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.385908 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.385969 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:12.386233 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:12.386373 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:12.386439 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:12.386571 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 12:09:12.386592 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260619 12:09:12.392604 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:12.389223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.0.203.188:34975)
0619 12:09:12.389277 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:12.389278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:12.389757 (+ 479us) server_negotiation.cc:407] Connection header received
0619 12:09:12.389839 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:12.389840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:12.389854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:12.389895 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:12.390026 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.390507 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.391717 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.392017 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.392096 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:12.392318 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:12.392354 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:12.392407 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:12.392491 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 12:09:12.392522 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:09:12.392650 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:12.389103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59448 (local address 127.0.203.190:42581)
0619 12:09:12.389355 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:12.389357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:12.389365 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:12.389418 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:12.389418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:12.389499 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:12.389592 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:12.389761 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.390571 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.391642 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.391938 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.392017 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:12.392221 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:12.392354 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:12.392414 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:12.392512 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 12:09:12.392541 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":66,"threads_started":1}
I20260619 12:09:12.392940 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:12.389227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.0.203.189:34895)
0619 12:09:12.389275 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:12.389277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:12.389436 (+ 159us) server_negotiation.cc:407] Connection header received
0619 12:09:12.389466 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:12.389467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:12.389481 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:12.389539 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:12.389746 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.390566 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.391792 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:12.392022 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:12.392085 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:12.392234 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:12.392283 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:12.392328 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:12.392659 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 12:09:12.392865 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:09:12.394694 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 23037
I20260619 12:09:12.399802 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 23112
I20260619 12:09:12.405191 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 23187
2026-06-19T12:09:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2828 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:12 dist-test-slave-44vj krb5kdc[23407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:12 dist-test-slave-44vj krb5kdc[23407](info): set up 2 sockets
Jun 19 12:09:12 dist-test-slave-44vj krb5kdc[23407](info): commencing operation
krb5kdc: starting...
W20260619 12:09:13.387984 814 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/test-user.keytab.
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.203.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
Entry for principal kudu/127.0.203.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab.
I20260619 12:09:13.512310 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:37605
--webserver_interface=127.0.203.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:37605,127.0.203.189:45429,127.0.203.188:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:13.599223 23420 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:09:13.599532 23420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:13.599556 23420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:13.599581 23420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:13.600849 23420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:13.600898 23420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:13.600910 23420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:13.600922 23420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:13.600934 23420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:13.602571 23420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:37605,127.0.203.189:45429,127.0.203.188:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.190.keytab
--principal=kudu/127.0.203.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.190:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.203.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:13.602799 23420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:13.602995 23420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:13.608479 23425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:13.608490 23428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:13.608637 23420 server_base.cc:1034] running on GCE node
W20260619 12:09:13.608490 23426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:13.609354 23420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:13.609757 23420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:13.610966 23420 hybrid_clock.cc:648] HybridClock initialized: now 1781870953610938 us; error 53 us; skew 500 ppm
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:13.613876 23420 init.cc:377] Logged in from keytab as kudu/127.0.203.190@KRBTEST.COM (short username kudu)
I20260619 12:09:13.614405 23420 webserver.cc:460] Webserver started at http://127.0.203.190:37385/ using document root <none> and password file <none>
I20260619 12:09:13.614636 23420 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:13.614688 23420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:13.614820 23420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:13.615957 23420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data/instance:
uuid: "1d0b0c8c719949fca02691008ada2846"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "15d7454e0544f000faa6c81453a566e7"
server_key_iv: "59680d4dfef40792bbf8b5600758a77d"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.616310 23420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/wal/instance:
uuid: "1d0b0c8c719949fca02691008ada2846"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "15d7454e0544f000faa6c81453a566e7"
server_key_iv: "59680d4dfef40792bbf8b5600758a77d"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.617753 23420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:13.618611 23435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:13.618844 23420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:13.618957 23420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/wal
uuid: "1d0b0c8c719949fca02691008ada2846"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "15d7454e0544f000faa6c81453a566e7"
server_key_iv: "59680d4dfef40792bbf8b5600758a77d"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.619045 23420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:13.644050 23420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:13.646538 23420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:13.646667 23420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:13.651214 23420 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.190:37605
I20260619 12:09:13.651281 23487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.190:37605 every 8 connection(s)
I20260619 12:09:13.651680 23420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:13.652751 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:13.654292 23488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:13.658594 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 23420
I20260619 12:09:13.658679 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-0/wal/instance
I20260619 12:09:13.658852 814 external_mini_cluster.cc:1428] Setting key 3ffd6f642f6eda2ad08ce23e798f4ccd
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:13.666249 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.655017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56306 (local address 127.0.203.190:37605)
0619 12:09:13.655188 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:13.655197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:13.655284 (+ 87us) server_negotiation.cc:407] Connection header received
0619 12:09:13.655872 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:13.655875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:13.655896 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:13.656014 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:13.656605 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.657547 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.658538 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.658982 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.659096 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.661838 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:13.661851 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:13.661851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:13.661863 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:13.664085 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.664619 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.664622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.664622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.664740 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.665097 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.665098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.665098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.665520 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:13.665628 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:13.665924 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 12:09:13.666064 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260619 12:09:13.666332 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.654958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:37605 (local address 127.0.0.1:56306)
0619 12:09:13.655156 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.655199 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:13.655678 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:13.656035 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:13.656038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:13.656291 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:13.656467 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.656471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.657650 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.657651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.658407 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.658410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.659109 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.659110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.659197 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.659837 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:13.659847 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:13.661715 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:13.664202 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.664212 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.664214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.664495 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.664837 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.664838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.664839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.664988 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.665628 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:13.665636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:13.665760 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 12:09:13.665881 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 12:09:13.666154 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":58,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.0.203.189@KRBTEST.COM; defaulting to no policy
I20260619 12:09:13.667522 23488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:13.667927 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.667770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:37990)
0619 12:09:13.667830 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.667896 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:09:13.668087 23436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
I20260619 12:09:13.668534 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.668443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:37992)
0619 12:09:13.668490 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.668515 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:09:13.668627 23488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:45429: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab.
Entry for principal kudu/127.0.203.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab.
I20260619 12:09:13.686404 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:45429
--webserver_interface=127.0.203.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab
--principal=kudu/127.0.203.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:37605,127.0.203.189:45429,127.0.203.188:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:09:13.717878 23488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } attempt: 1
I20260619 12:09:13.718573 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.718384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:38002)
0619 12:09:13.718460 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.718533 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:09:13.719002 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.718901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:38008)
0619 12:09:13.718949 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.718981 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 12:09:13.719125 23488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:45429: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
W20260619 12:09:13.776199 23495 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:09:13.776479 23495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:13.776496 23495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:13.776520 23495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:13.777824 23495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:13.777879 23495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:13.777892 23495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:13.777905 23495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:13.777915 23495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:13.779633 23495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:37605,127.0.203.189:45429,127.0.203.188:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.189.keytab
--principal=kudu/127.0.203.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.189:45429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.203.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:13.779915 23495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:13.780167 23495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:13.785233 23503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:13.785233 23501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:13.785347 23500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:13.786063 23495 server_base.cc:1034] running on GCE node
I20260619 12:09:13.786332 23495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:13.786646 23495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:13.787297 23488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } attempt: 2
I20260619 12:09:13.787806 23495 hybrid_clock.cc:648] HybridClock initialized: now 1781870953787794 us; error 34 us; skew 500 ppm
I20260619 12:09:13.787981 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.787789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:38014)
0619 12:09:13.787886 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.787934 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 12:09:13.788486 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.788296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:38022)
0619 12:09:13.788395 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.788444 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260619 12:09:13.788636 23488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.189:45429: Network error: Client connection negotiation failed: client connection to 127.0.203.189:45429: connect: Connection refused (error 111)
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:13.790725 23495 init.cc:377] Logged in from keytab as kudu/127.0.203.189@KRBTEST.COM (short username kudu)
I20260619 12:09:13.791251 23495 webserver.cc:460] Webserver started at http://127.0.203.189:37911/ using document root <none> and password file <none>
I20260619 12:09:13.791512 23495 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:13.791563 23495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:13.791702 23495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:13.792886 23495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data/instance:
uuid: "1da825c1aa644d3997559d07ec0b619a"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "80291fc06c5246a6a0cf42cba95024fa"
server_key_iv: "61d6cd6b775489c20af48cc6653c71ad"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.793234 23495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/wal/instance:
uuid: "1da825c1aa644d3997559d07ec0b619a"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "80291fc06c5246a6a0cf42cba95024fa"
server_key_iv: "61d6cd6b775489c20af48cc6653c71ad"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.794721 23495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:13.795603 23510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:13.795823 23495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:13.795919 23495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/wal
uuid: "1da825c1aa644d3997559d07ec0b619a"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "80291fc06c5246a6a0cf42cba95024fa"
server_key_iv: "61d6cd6b775489c20af48cc6653c71ad"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.795989 23495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:13.809293 23495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:13.812076 23495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:13.812222 23495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:13.817170 23495 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.189:45429
I20260619 12:09:13.817348 23562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.189:45429 every 8 connection(s)
I20260619 12:09:13.817634 23495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/data/info.pb
I20260619 12:09:13.818843 23563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:13.820677 23563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:13.822492 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 23495
I20260619 12:09:13.822603 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-1/wal/instance
I20260619 12:09:13.822788 814 external_mini_cluster.cc:1428] Setting key aa0335ea46786c8c8ae568e1837a0ed0
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.203.188@KRBTEST.COM; defaulting to no policy
I20260619 12:09:13.833066 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.821697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56308 (local address 127.0.203.190:37605)
0619 12:09:13.821860 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:13.821862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:13.822088 (+ 226us) server_negotiation.cc:407] Connection header received
0619 12:09:13.822902 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:13.822904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:13.822925 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:13.823001 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:13.823946 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.824515 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.825939 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.826222 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.826342 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.829476 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:13.829478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:13.829478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:13.829488 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:13.831234 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.831820 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.831821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.831822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.831888 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.832234 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.832235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.832236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.832392 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:13.832462 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:13.832892 (+ 430us) server_negotiation.cc:299] Negotiation successful
0619 12:09:13.832987 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260619 12:09:13.833216 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.821625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:37605 (local address 127.0.0.1:56308)
0619 12:09:13.821890 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.821943 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:13.822743 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:13.823107 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:13.823112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:13.823551 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:13.823835 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.823841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.824627 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.824628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.825791 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.825795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.826369 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.826370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.826559 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.827225 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:13.827238 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:13.829314 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:13.831355 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.831409 (+ 54us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.831412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.831736 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.831995 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.831996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.831996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.832141 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.832474 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:13.832480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:13.832692 (+ 212us) client_negotiation.cc:769] Sending connection context
0619 12:09:13.832799 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:09:13.832984 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":131,"threads_started":1}
I20260619 12:09:13.833963 23563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.203.188@KRBTEST.COM" created.
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:13.843782 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.834296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:38032)
0619 12:09:13.834356 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.834372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:13.834415 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:13.834804 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:13.834805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:13.834985 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:13.835074 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.835076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.836154 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.836155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.836915 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.836918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.837766 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.837768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.837859 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.838469 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:13.838475 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:13.840075 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:13.842324 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.842325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.842326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.842556 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.842852 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.842853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.842853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.842928 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.843405 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:13.843406 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:13.843469 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 12:09:13.843553 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:09:13.843675 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:09:13.843931 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.834384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.0.203.189:45429)
0619 12:09:13.834615 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:13.834620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:13.834643 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:09:13.834670 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:13.834673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:13.834691 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:13.834772 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:13.835188 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.836051 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.837026 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.837539 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.837701 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.840223 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:13.840240 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:13.840241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:13.840250 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:13.842204 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.842689 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.842692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.842693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.842770 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.843042 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.843044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.843044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.843246 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:13.843332 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:13.843595 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 12:09:13.843780 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":195,"thread_start_us":142,"threads_started":1}
I20260619 12:09:13.844734 23563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:13.845165 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.845009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47828)
0619 12:09:13.845063 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.845130 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:09:13.845338 23512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
I20260619 12:09:13.845908 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.845774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47840)
0619 12:09:13.845811 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.845879 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 12:09:13.846117 23563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:37775: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.203.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab.
Entry for principal kudu/127.0.203.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab.
I20260619 12:09:13.850728 814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskgwdYux/build/release/bin/kudu
/tmp/dist-test-taskgwdYux/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:37775
--webserver_interface=127.0.203.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab
--principal=kudu/127.0.203.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.203.148:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.203.190:37605,127.0.203.189:45429,127.0.203.188:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:09:13.879808 23488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } attempt: 3
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:13.888008 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.880392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:38036)
0619 12:09:13.880482 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.880498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:13.880578 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:13.880759 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:13.880760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:13.880915 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:13.880999 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.881000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.881995 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.881996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.882595 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.882597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.883111 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.883112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.883171 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.883723 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:13.883727 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:13.884913 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:13.886805 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.886806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.886807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.887018 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.887253 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.887254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.887254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.887315 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.887676 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:13.887677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:13.887737 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 12:09:13.887821 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:09:13.887927 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 12:09:13.888068 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.880477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.0.203.189:45429)
0619 12:09:13.880575 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:13.880578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:13.880621 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:09:13.880660 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:13.880661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:13.880679 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:13.880751 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:13.881080 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.881899 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.882684 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.883040 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.883113 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.885016 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:13.885018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:13.885018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:13.885027 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:13.886696 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.887100 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.887101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.887101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.887167 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.887404 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.887405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.887405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.887552 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:13.887625 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:13.887855 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 12:09:13.887958 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:09:13.888564 23488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:13.888953 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.888843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47848)
0619 12:09:13.888894 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.888927 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:09:13.889259 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.889183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47850)
0619 12:09:13.889217 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.889239 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 12:09:13.889336 23488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:37775: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
I20260619 12:09:13.895423 23563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } attempt: 1
I20260619 12:09:13.895993 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.895832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47856)
0619 12:09:13.895905 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.895960 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 12:09:13.896451 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.896297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47872)
0619 12:09:13.896358 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.896401 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:09:13.896598 23563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:37775: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
I20260619 12:09:13.920519 23488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } attempt: 1
I20260619 12:09:13.921067 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.920924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47876)
0619 12:09:13.920993 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.921028 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 12:09:13.921563 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.921304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47878)
0619 12:09:13.921411 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.921500 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 12:09:13.921751 23488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:37775: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
W20260619 12:09:13.942368 23570 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:09:13.942623 23570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:13.942641 23570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:13.942662 23570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:13.944011 23570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:13.944061 23570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:13.944075 23570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:13.944087 23570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:13.944097 23570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:13.945781 23570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.203.148:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.203.190:37605,127.0.203.189:45429,127.0.203.188:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/krb5kdc/kudu_127.0.203.188.keytab
--principal=kudu/127.0.203.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.203.188:37775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.203.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:13.946061 23570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:13.946323 23570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:13.951714 23578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:13.951761 23570 server_base.cc:1034] running on GCE node
W20260619 12:09:13.951676 23575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:13.951711 23576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:13.952224 23570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:13.952564 23570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:13.953748 23570 hybrid_clock.cc:648] HybridClock initialized: now 1781870953953737 us; error 62 us; skew 500 ppm
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:13.956611 23570 init.cc:377] Logged in from keytab as kudu/127.0.203.188@KRBTEST.COM (short username kudu)
I20260619 12:09:13.957135 23570 webserver.cc:460] Webserver started at http://127.0.203.188:46629/ using document root <none> and password file <none>
I20260619 12:09:13.957463 23570 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:13.957531 23570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:13.957674 23570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:13.958859 23570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data/instance:
uuid: "cf2b8995043c4be5a7cfade77c97790a"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "3c50cd86ca63a1d4f457cd1cef1297eb"
server_key_iv: "c91c4123a4b4fb50da803158f82ab4f2"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.959451 23570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/wal/instance:
uuid: "cf2b8995043c4be5a7cfade77c97790a"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "3c50cd86ca63a1d4f457cd1cef1297eb"
server_key_iv: "c91c4123a4b4fb50da803158f82ab4f2"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.961652 23570 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:13.962663 23585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:13.962950 23570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:13.963076 23570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data,/tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/wal
uuid: "cf2b8995043c4be5a7cfade77c97790a"
format_stamp: "Formatted at 2026-06-19 12:09:13 on dist-test-slave-44vj"
server_key: "3c50cd86ca63a1d4f457cd1cef1297eb"
server_key_iv: "c91c4123a4b4fb50da803158f82ab4f2"
server_key_version: "encryptionkey@0"
I20260619 12:09:13.963153 23570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:13.964766 23563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } attempt: 2
I20260619 12:09:13.965378 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.965204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47890)
0619 12:09:13.965279 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.965340 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 12:09:13.965757 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.965656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47896)
0619 12:09:13.965704 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.965736 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 12:09:13.965874 23563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.203.188:37775: Network error: Client connection negotiation failed: client connection to 127.0.203.188:37775: connect: Connection refused (error 111)
I20260619 12:09:13.979266 23570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:13.981499 23570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:13.981650 23570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:13.986248 23570 rpc_server.cc:307] RPC server started. Bound to: 127.0.203.188:37775
I20260619 12:09:13.986356 23637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.203.188:37775 every 8 connection(s)
I20260619 12:09:13.986905 23570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/data/info.pb
I20260619 12:09:13.988166 23638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:13.990149 23638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
I20260619 12:09:13.996930 23488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } attempt: 2
I20260619 12:09:13.997031 814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskgwdYux/build/release/bin/kudu as pid 23570
I20260619 12:09:13.997126 814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskgwdYux/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870673623844-814-0/minicluster-data/master-2/wal/instance
I20260619 12:09:13.997331 814 external_mini_cluster.cc:1428] Setting key 167ae7ace0498bfede7de736c538bdc1
I20260619 12:09:14.000938 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.991112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.0.203.190:37605)
0619 12:09:13.991198 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:13.991201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:13.991400 (+ 199us) server_negotiation.cc:407] Connection header received
0619 12:09:13.991982 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:13.991983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:13.991995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:13.992098 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:13.992858 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.993355 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.994490 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.994694 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.994771 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.997591 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:13.997593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:13.997594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:13.997601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:13.999285 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:13.999698 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:13.999700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:13.999700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:13.999754 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.000067 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.000069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.000069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.000272 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.000354 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.000720 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.000828 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:09:14.001117 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.991013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.190:37605 (local address 127.0.0.1:56316)
0619 12:09:13.991231 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.991281 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:13.991854 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:13.992197 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:13.992201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:13.992494 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:13.992752 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.992757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.993449 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.993450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.994384 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.994387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.994787 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.994787 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:13.994910 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:13.995516 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:13.995526 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:13.997245 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:13.999378 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.999385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.999388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.999628 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:13.999808 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:13.999809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:13.999809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:13.999946 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.000355 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:14.000362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:14.000541 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 12:09:14.000662 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 12:09:14.000862 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260619 12:09:14.001760 23638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:09:13 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.190@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
Jun 19 12:09:14 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
Jun 19 12:09:14 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:14.007341 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.997296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47908)
0619 12:09:13.997368 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:13.997380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:13.997417 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:13.997907 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:13.997910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:13.998097 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:13.998161 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:13.998163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.999869 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:13.999870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.000612 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:14.000615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.001268 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.001269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.001332 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.001950 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:14.001954 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:14.003518 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:14.006167 (+ 2649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.006168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.006169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.006354 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.006628 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.006628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.006629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.006698 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.006996 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:14.006997 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:14.007049 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 12:09:14.007114 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:09:14.007236 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:09:14.007567 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.997714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.0.203.188:37775)
0619 12:09:13.997966 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:13.997968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:13.997975 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:13.998040 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:13.998040 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:13.998081 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:13.998512 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:13.998993 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.999809 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.000786 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.001316 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.001386 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.004194 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.004196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.004196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.004204 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.006018 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.006440 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.006444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.006444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.006528 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.006759 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.006761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.006761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.006972 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.007153 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.007346 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.007453 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":206,"thread_start_us":114,"threads_started":1}
I20260619 12:09:14.008370 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:13.997395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.0.203.188:37775)
0619 12:09:13.997643 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:13.997650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:13.997698 (+ 48us) server_negotiation.cc:407] Connection header received
0619 12:09:13.997733 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:13.997736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:13.997754 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:13.998240 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:13.998601 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:13.999785 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.000699 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.001164 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.001482 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.003628 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.003636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.003636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.003643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.006060 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.006440 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.006444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.006444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.006546 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.006763 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.006764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.006765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.006930 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.007005 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.008143 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.008240 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":132,"threads_started":1}
I20260619 12:09:14.010448 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.002068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.189:45429 (local address 127.0.0.1:38038)
0619 12:09:14.002183 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:14.002195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:14.002252 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:14.002411 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:14.002412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:14.002557 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:14.002620 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:14.002621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.003264 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.003265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.003935 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:14.003938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.004397 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.004398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.004473 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.005447 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:14.005451 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:14.006992 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:14.009383 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.009385 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.009385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.009554 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.009762 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.009763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.009763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.009810 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.010140 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:14.010141 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:14.010202 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 12:09:14.010277 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:09:14.010372 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260619 12:09:14.010511 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.002134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.0.203.189:45429)
0619 12:09:14.002203 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.002206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.002244 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:09:14.002312 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.002314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.002328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.002402 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.002697 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.003204 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.004004 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.004273 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.004340 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.007138 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.007139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.007140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.007147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.009294 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.009630 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.009631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.009631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.009696 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.009870 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.009871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.009871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.010024 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.010093 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.010324 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.010420 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:09:14.010630 23488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846: Bootstrap starting.
I20260619 12:09:14.011413 23488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:14.011775 23488 log.cc:824] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:14.011986 23638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:14.012843 23488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846: No bootstrap required, opened a new log
I20260619 12:09:14.015012 23488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
I20260619 12:09:14.015201 23488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d0b0c8c719949fca02691008ada2846, State: Initialized, Role: FOLLOWER
I20260619 12:09:14.015286 23488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
I20260619 12:09:14.015709 23488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:14.015731 23645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.015986 23645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 12:09:14 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.190@KRBTEST.COM
Jun 19 12:09:14 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.188@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
W20260619 12:09:14.018355 23656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:14.018463 23656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:14.020195 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.012466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47934)
0619 12:09:14.012508 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:14.012519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:14.012574 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:14.012719 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:14.012720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:14.012870 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:14.012921 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:14.012924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.013793 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.013794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.014881 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:14.014884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.015327 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.015327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.015410 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.015828 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:14.015833 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:14.017274 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:14.019009 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.019010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.019011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.019206 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.019474 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.019475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.019475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.019529 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.019852 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:14.019853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:14.019892 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 12:09:14.019942 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:09:14.020061 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 12:09:14.020469 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.012511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47934 (local address 127.0.203.188:37775)
0619 12:09:14.012554 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.012556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.012563 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:14.012631 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.012632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.012641 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.012707 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.013013 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.013707 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.014968 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.015247 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.016393 (+ 1146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.017368 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.017370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.017370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.017378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.018924 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.019278 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.019279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.019279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.019330 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.019581 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.019582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.019582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.019774 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.019829 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.020322 (+ 493us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.020396 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:09:14.019083 23420 master_runner.cc:186] Error getting master registration for 127.0.203.189:45429: OK, instance_id { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" instance_seqno: 1781870953815716 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:09:14.022148 23420 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:09:14.024037 23638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: Bootstrap starting.
I20260619 12:09:14.024426 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.011677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.0.203.190:37605)
0619 12:09:14.011724 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.011725 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.011734 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:14.011761 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.011761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.011772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.011831 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.012583 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.013210 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.013891 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.014091 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.014147 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.018040 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.018041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.018041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.018049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.022952 (+ 4903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.023521 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.023522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.023523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.023590 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.023833 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.023833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.023834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.023984 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.024134 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.024252 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.024338 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:09:14.024947 23638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:14.025297 23638 log.cc:824] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:14.026392 23638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: No bootstrap required, opened a new log
I20260619 12:09:14.028740 23638 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
I20260619 12:09:14.028972 23638 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2b8995043c4be5a7cfade77c97790a, State: Initialized, Role: FOLLOWER
I20260619 12:09:14.029103 23638 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
Jun 19 12:09:14 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.203.189@KRBTEST.COM
I20260619 12:09:14.029482 23659 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.029584 23638 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:14.029624 23659 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:09:14.032835 23670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:14.032893 23670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:14.032985 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.025236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.0.203.189:45429)
0619 12:09:14.025286 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.025288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.025296 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:14.025324 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.025325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.025337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.025389 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.025964 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.026530 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.027176 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.027454 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.027522 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.029871 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.029873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.029873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.029880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.031515 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.031876 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.031877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.031878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.031935 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.032131 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.032131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.032132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.032282 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.032420 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.032800 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.032888 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:09:14.034189 814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:14.035177 23462 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1d0b0c8c719949fca02691008ada2846"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56328
I20260619 12:09:14.035336 23462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 12:09:14.035405 23462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:14.033411 23570 master_runner.cc:186] Error getting master registration for 127.0.203.189:45429: OK, instance_id { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" instance_seqno: 1781870953815716 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:09:14.035907 23570 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:09:14.036363 23462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
I20260619 12:09:14.036576 23462 leader_election.cc:288] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [CANDIDATE]: Term 1 election: Requested vote from peers 1da825c1aa644d3997559d07ec0b619a (127.0.203.189:45429), cf2b8995043c4be5a7cfade77c97790a (127.0.203.188:37775)
I20260619 12:09:14.036877 23612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d0b0c8c719949fca02691008ada2846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cf2b8995043c4be5a7cfade77c97790a"
I20260619 12:09:14.036998 23612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:14.037989 23612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d0b0c8c719949fca02691008ada2846 in term 1.
I20260619 12:09:14.038224 23437 leader_election.cc:302] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d0b0c8c719949fca02691008ada2846, cf2b8995043c4be5a7cfade77c97790a; no voters:
I20260619 12:09:14.038417 23645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:14.038646 23645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [term 1 LEADER]: Becoming Leader. State: Replica: 1d0b0c8c719949fca02691008ada2846, State: Running, Role: LEADER
I20260619 12:09:14.038767 23645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
I20260619 12:09:14.039305 23677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d0b0c8c719949fca02691008ada2846. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.039508 23677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:14.039867 23679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:14.040236 23679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:14.042398 23612 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [term 1 FOLLOWER]: Refusing update from remote peer 1d0b0c8c719949fca02691008ada2846: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:14.042855 23677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:14.044927 23659 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d0b0c8c719949fca02691008ada2846. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.045070 23659 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:14.055320 23645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.055991 23645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:14.056463 23680 mvcc.cc:205] Tried to move back new op lower bound from 7298543427754250240 to 7298543427743764480. Current Snapshot: MvccSnapshot[applied={T|T < 7298543427754250240}]
I20260619 12:09:14.057008 23563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } attempt: 3
I20260619 12:09:14.057323 23659 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.057430 23659 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:14.057650 23681 mvcc.cc:205] Tried to move back new op lower bound from 7298543427754250240 to 7298543427743764480. Current Snapshot: MvccSnapshot[applied={T|T < 7298543427754250240}]
I20260619 12:09:14.058375 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.047692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.0.203.190:37605)
0619 12:09:14.047762 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.047764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.047979 (+ 215us) server_negotiation.cc:407] Connection header received
0619 12:09:14.048185 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.048187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.048200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.048233 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.049534 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.050292 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.051927 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.052226 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.052268 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.055194 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.055208 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.055208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.055217 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.056955 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.057468 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.057470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.057470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.057530 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.057753 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.057754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.057754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.057883 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.057973 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.058195 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.058278 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:09:14.059983 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.037719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.0.203.189:45429)
0619 12:09:14.037771 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.037773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.047751 (+ 9978us) server_negotiation.cc:407] Connection header received
0619 12:09:14.047941 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.047943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.047958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.047991 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.049359 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.050175 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.051948 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.052278 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.052322 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.055422 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.055424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.055425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.055444 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.057062 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.058423 (+ 1361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.058424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.058425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.058486 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.059147 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.059148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.059149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.059294 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.059423 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.059792 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.059880 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 12:09:14 dist-test-slave-44vj krb5kdc[23407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.203.189@KRBTEST.COM for kudu/127.0.203.188@KRBTEST.COM
I20260619 12:09:14.063424 23679 catalog_manager.cc:1347] Generated new cluster ID: 2e03c8f4f7c44d0aaa2edf306885b9c1
I20260619 12:09:14.063489 23679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:14.071558 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.047984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.0.203.188:37775)
0619 12:09:14.048031 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.048033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.051528 (+ 3495us) server_negotiation.cc:407] Connection header received
0619 12:09:14.051628 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.051629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.051642 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.051670 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.052413 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.053097 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.054834 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.056746 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.056825 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.067777 (+ 10952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.067780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.067780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.067789 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.069503 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.070075 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.070076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.070077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.070168 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.070419 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.070420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.070421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.070596 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.070710 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.071315 (+ 605us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.071439 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:14.077819 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.057317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.203.188:37775 (local address 127.0.0.1:47948)
0619 12:09:14.057403 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:14.057414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:14.057449 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:14.057681 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:14.057682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:14.057841 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:14.057891 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:14.057893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.058721 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.058722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.059437 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:14.059440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.059963 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.059964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:14.060023 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.060376 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:14.060381 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:14.061883 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:14.076349 (+ 14466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.076351 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.076352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.076718 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.077009 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:14.077010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:14.077011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:14.077087 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:14.077434 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:14.077435 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:14.077507 (+ 72us) client_negotiation.cc:769] Sending connection context
0619 12:09:14.077567 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:09:14.077697 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 12:09:14.078455 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:14.057515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.0.203.188:37775)
0619 12:09:14.057558 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.057559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.057566 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:14.057584 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.057585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.057594 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.057637 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.057977 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.058658 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.059523 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:14.059855 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:14.059913 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:14.061981 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:14.061983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:14.061983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:14.061991 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:14.076202 (+ 14211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.076839 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.076842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.076842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.076919 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:14.077156 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:14.077157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:14.077157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:14.077329 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:14.077389 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:14.078255 (+ 866us) server_negotiation.cc:299] Negotiation successful
0619 12:09:14.078356 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:14.080404 23563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a: Bootstrap starting.
I20260619 12:09:14.081182 23563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:14.081504 23563 log.cc:824] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:14.081748 23679 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:14.082446 23679 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:14.088434 23563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a: No bootstrap required, opened a new log
I20260619 12:09:14.090348 23563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
I20260619 12:09:14.090498 23563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da825c1aa644d3997559d07ec0b619a, State: Initialized, Role: FOLLOWER
I20260619 12:09:14.090566 23563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } }
I20260619 12:09:14.090921 23563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:14.091081 23536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:14.092386 23536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [term 1 FOLLOWER]: Refusing update from remote peer 1d0b0c8c719949fca02691008ada2846: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:14.092905 23677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:14.093149 23537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d0b0c8c719949fca02691008ada2846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1da825c1aa644d3997559d07ec0b619a"
I20260619 12:09:14.094023 23537 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d0b0c8c719949fca02691008ada2846 in term 1.
I20260619 12:09:14.097041 23679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846: Generated new TSK 0
I20260619 12:09:14.097266 23679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:14.098300 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.098591 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:14.098690 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d0b0c8c719949fca02691008ada2846. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.098805 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:14.098870 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.098960 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:14.099793 23699 catalog_manager.cc:1259] Loaded cluster ID: 2e03c8f4f7c44d0aaa2edf306885b9c1
I20260619 12:09:14.099879 23699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a: loading cluster ID for follower catalog manager: success
I20260619 12:09:14.101183 23699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a: acquiring CA information for follower catalog manager: success
W20260619 12:09:14.101382 23699 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260619 12:09:14.104269 23645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d0b0c8c719949fca02691008ada2846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0b0c8c719949fca02691008ada2846" member_type: VOTER last_known_addr { host: "127.0.203.190" port: 37605 } } peers { permanent_uuid: "1da825c1aa644d3997559d07ec0b619a" member_type: VOTER last_known_addr { host: "127.0.203.189" port: 45429 } } peers { permanent_uuid: "cf2b8995043c4be5a7cfade77c97790a" member_type: VOTER last_known_addr { host: "127.0.203.188" port: 37775 } } }
I20260619 12:09:14.104379 23645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d0b0c8c719949fca02691008ada2846 [sys.catalog]: This master's current role is: LEADER
W20260619 12:09:14.120580 23491 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.118951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.0.203.190:37605)
0619 12:09:14.118995 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.118997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.119925 (+ 928us) server_negotiation.cc:407] Connection header received
0619 12:09:14.120027 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.120029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.120040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.120070 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.120496 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56344: BlockingRecv error: recv got EOF from 127.0.0.1:56344 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 12:09:14.121311 23643 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.119309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.0.203.188:37775)
0619 12:09:14.119352 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.119354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.120821 (+ 1467us) server_negotiation.cc:407] Connection header received
0619 12:09:14.120908 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.120909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.120920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.120947 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.121250 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47962: BlockingRecv error: recv got EOF from 127.0.0.1:47962 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 12:09:14.122072 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.119671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38060 (local address 127.0.203.189:45429)
0619 12:09:14.119715 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.119717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.121519 (+ 1802us) server_negotiation.cc:407] Connection header received
0619 12:09:14.121620 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.121621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.121632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.121657 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.121968 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38060: BlockingRecv error: recv got EOF from 127.0.0.1:38060 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 12:09:14.124423 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.123673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38070 (local address 127.0.203.189:45429)
0619 12:09:14.123705 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.123707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.123948 (+ 241us) server_negotiation.cc:407] Connection header received
0619 12:09:14.124042 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.124043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.124063 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.124090 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.124379 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38070: BlockingRecv error: recv got EOF from 127.0.0.1:38070 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 12:09:14.127516 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.126731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.0.203.189:45429)
0619 12:09:14.126764 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.126766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.127002 (+ 236us) server_negotiation.cc:407] Connection header received
0619 12:09:14.127089 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.127090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.127100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.127126 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.127467 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38082: BlockingRecv error: recv got EOF from 127.0.0.1:38082 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 12:09:14.131762 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.130798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.0.203.189:45429)
0619 12:09:14.130847 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.130849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.131103 (+ 254us) server_negotiation.cc:407] Connection header received
0619 12:09:14.131200 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.131201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.131212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.131241 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.131708 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38084: BlockingRecv error: recv got EOF from 127.0.0.1:38084 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 12:09:14.137977 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.137109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.0.203.189:45429)
0619 12:09:14.137152 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.137154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.137385 (+ 231us) server_negotiation.cc:407] Connection header received
0619 12:09:14.137479 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.137480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.137491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.137520 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.137916 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38100: BlockingRecv error: recv got EOF from 127.0.0.1:38100 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 12:09:14.149082 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.147842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.0.203.189:45429)
0619 12:09:14.148133 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.148135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.148446 (+ 311us) server_negotiation.cc:407] Connection header received
0619 12:09:14.148570 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.148571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.148584 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.148615 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.149002 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38104: BlockingRecv error: recv got EOF from 127.0.0.1:38104 (error 108)
Metrics: {"server-negotiator.queue_time_us":258}
W20260619 12:09:14.168013 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.166825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.0.203.189:45429)
0619 12:09:14.166878 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.166881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.167192 (+ 311us) server_negotiation.cc:407] Connection header received
0619 12:09:14.167319 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.167321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.167335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.167405 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.167934 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38112: BlockingRecv error: recv got EOF from 127.0.0.1:38112 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 12:09:14.202689 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.201701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.0.203.189:45429)
0619 12:09:14.201754 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.201757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.202032 (+ 275us) server_negotiation.cc:407] Connection header received
0619 12:09:14.202136 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.202137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.202150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.202180 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.202613 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38116: BlockingRecv error: recv got EOF from 127.0.0.1:38116 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 12:09:14.269053 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.268348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.0.203.189:45429)
0619 12:09:14.268443 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.268445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.268457 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:09:14.268524 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.268525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.268540 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.268592 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.268971 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38130: BlockingRecv error: recv got EOF from 127.0.0.1:38130 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 12:09:14.399564 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.398518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.0.203.189:45429)
0619 12:09:14.398616 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.398619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.398762 (+ 143us) server_negotiation.cc:407] Connection header received
0619 12:09:14.398889 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.398890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.398902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.398963 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.399484 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38140: BlockingRecv error: recv got EOF from 127.0.0.1:38140 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 12:09:14.657804 23567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:14.657005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.0.203.189:45429)
0619 12:09:14.657084 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:14.657086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:14.657141 (+ 55us) server_negotiation.cc:407] Connection header received
0619 12:09:14.657209 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:14.657210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:14.657222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:14.657281 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:14.657743 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38154: BlockingRecv error: recv got EOF from 127.0.0.1:38154 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:09:15.033354 23670 catalog_manager.cc:1259] Loaded cluster ID: 2e03c8f4f7c44d0aaa2edf306885b9c1
I20260619 12:09:15.033455 23670 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: loading cluster ID for follower catalog manager: success
I20260619 12:09:15.034235 23670 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: acquiring CA information for follower catalog manager: success
I20260619 12:09:15.034519 23670 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf2b8995043c4be5a7cfade77c97790a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:09:15.102035 23699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1da825c1aa644d3997559d07ec0b619a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:09:15.173650 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:15.171150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.0.203.189:45429)
0619 12:09:15.171325 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:15.171328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:15.171336 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:15.171379 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:15.171379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:15.171433 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:15.171504 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:15.171662 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.172162 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.172906 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.173105 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.173159 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:15.173316 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:15.173425 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:15.173495 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:15.173535 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 12:09:15.173558 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260619 12:09:15.178120 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:15.175682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.0.203.188:37775)
0619 12:09:15.175816 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:15.175818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:15.175892 (+ 74us) server_negotiation.cc:407] Connection header received
0619 12:09:15.175992 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:15.175993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:15.176028 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:15.176096 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:15.176196 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.176692 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.177411 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.177607 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.177661 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:15.177802 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:15.177908 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:15.177971 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:15.178007 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 12:09:15.178030 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260619 12:09:15.182996 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:15.179902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.0.203.189:45429)
0619 12:09:15.179946 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:15.179948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:15.180182 (+ 234us) server_negotiation.cc:407] Connection header received
0619 12:09:15.180213 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:15.180214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:15.180226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:15.180286 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:15.180470 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.181244 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.182258 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.182496 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.182554 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:15.182725 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:15.182761 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:15.182787 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:15.182890 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:09:15.182915 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:15.183053 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:15.180190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.0.203.188:37775)
0619 12:09:15.180337 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:15.180339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:15.180345 (+ 6us) server_negotiation.cc:407] Connection header received
0619 12:09:15.180366 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:15.180367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:15.180375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:15.180417 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:15.180557 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.181123 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.182184 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.182482 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.182543 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:15.182678 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:15.182717 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:15.182761 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:15.182820 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:09:15.182846 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260619 12:09:15.183796 23786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:15.180053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.0.203.190:37605)
0619 12:09:15.180197 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:15.180200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:15.180448 (+ 248us) server_negotiation.cc:407] Connection header received
0619 12:09:15.180507 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:15.180508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:15.180636 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:15.180743 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:15.180887 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.181543 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.182438 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:15.182759 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:15.182825 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:15.183222 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:15.183538 (+ 316us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:15.183612 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:15.183674 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 12:09:15.183709 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260619 12:09:15.189504 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 23420
I20260619 12:09:15.195060 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 23495
I20260619 12:09:15.200440 814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskgwdYux/build/release/bin/kudu with pid 23570
2026-06-19T12:09:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2793 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5622 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (281581 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST